diff --git a/docs/decisions/0055-python-foundry-hosting-contracts.md b/docs/decisions/0055-python-foundry-hosting-contracts.md index 2997fc3f34f..dffb392bc76 100644 --- a/docs/decisions/0055-python-foundry-hosting-contracts.md +++ b/docs/decisions/0055-python-foundry-hosting-contracts.md @@ -37,18 +37,20 @@ Choose separate agent and native workflow hosts. Share protocol internals, but k configuration, and state ownership specific to each execution model. The decisions below define the intended hosting contracts, not the implementation status of each feature. +See the [Foundry Hosting README](../../python/packages/foundry_hosting/README.md) for the current usage contract. ### Host boundaries Use `ResponsesHostServer` and `InvocationsHostServer` for agents, and `ResponsesWorkflowHostServer` and `InvocationsWorkflowHostServer` for native workflows. Agent hosts reject `WorkflowAgent`; workflow hosts require a factory returning a native `Workflow`. -Do not silently unwrap agents, migrate their state, or retain a combined-host compatibility shim. +Do not silently unwrap agents or migrate their state. Workflow hosts expose checkpoint storage rather than agent-session storage or agent history settings. -The Responses workflow host owns native execution directly. Share the Invocations engine between its -public hosts rather than duplicating it. Group each protocol into agent, workflow, and shared modules, -with cross-protocol request contracts and factory resolution in a central shared module. +Each of the four hosts owns its own construction, execution, and validation. Do not route agent and workflow +execution through a shared host with mode switches; that couples their validation rules. Group each protocol into +agent and workflow modules. Shared modules hold only protocol-neutral utilities, such as request contracts, +factory resolution, and SSE framing. Keep all hosts in one package. Responses is the initial stable-surface target; Invocations and Toolbox remain experimental. Separate classes do not require separate distribution packages. @@ -56,10 +58,10 @@ remain experimental. Separate classes do not require separate distribution packa ### Request adaptation and construction Use `parse_input` for custom input adaptation across both protocols. Agent parsers return `InvocationRun`; -workflow parsers return `WorkflowTurn` containing either typed start input or complete pending replies. -Parsing does not replace the host's checkpoint and pending-authority validation. +workflow parsers return `WorkflowTurn` containing either typed start input or pending replies. Workflow hosts +always require a parser; there is no default input conversion. +Parsing does not replace the host's checkpoint binding or the workflow runtime's reply validation. Use `ResponsesRequest` for the Responses wrapper, including factory and option-hook inputs. -Do not retain aliases for the former parser keywords or request-type name. Agent hosts accept an instance or a sync/async factory implementing a named-request callable protocol. Factories must declare a required `request` parameter and are called as `factory(request=request)`. @@ -73,19 +75,31 @@ before `prepare_options`. Invocations agent factories receive the Starlette `Req and option preparation. Preserve these protocol-specific timings rather than forcing identical signatures or inventing a shared options-precedence policy. +By default, agent hosts forward only the caller's messages. Do not forward caller generation settings; the +agent's own `default_options` apply unchanged, together with host-owned storage/background controls. An +optional `prepare_options` callable on agent hosts is the opt-in customization path; forward only its returned, +validated mapping. This does not prevent application code from inspecting the original request. Responses +agent factories see translated options before the hook. Workflow hosts have no option hook: the parser forwards +model options explicitly through `WorkflowTurn` client kwargs. + Native workflow factories are request-aware and must allocate fresh workflows, executors, agents, clients, providers, and mutable tools. A new graph wrapper around shared executors is not isolation. Reject known shared execution resources rather than cloning them. This guard cannot prove that arbitrary application globals are isolated. Keep workflow names and executor IDs stable for checkpoint restoration. -Close factory-owned context-managed agents for the request, including failure and cancellation cleanup. -Applications own cleanup of other resources they create; caller-owned shared resources must not be closed. -State needed across requests belongs in supported stores, not request-owned instance fields. +Unwind successfully entered factory-agent contexts when a request finishes, fails, or is cancelled. +`Agent` also unwinds client and MCP contexts already entered if a later context fails to enter. +Allocations before agent entry and partial allocations from a failed client `__aenter__` may not be +cleaned up; calling `agent.close()` before entry does not dispose an unentered client. +Applications own cleanup of other resources they create. An entered caller-supplied context-managed +client can be closed on agent exit; ownership depends on its context contract, and the host cannot +reliably identify shared clients on first use. State needed across requests belongs in supported stores, +not request-owned instance fields. ### Persistence and history ownership Regular agents continue through agent-session storage. Native workflows continue through their exact -scoped checkpoints, including pending-reply authority. Neither execution model converts the other's state. +scoped checkpoints, including pending `request_info` requests. Neither execution model converts the other's state. Native workflow agents use checkpoint-owned history, not private inference-service continuation. Keep three storage decisions distinct: @@ -96,20 +110,34 @@ Keep three storage decisions distinct: Use `history_source="host_managed"` by default, with `"agent_managed"` and `"inference_service"` as explicit alternatives. `agent_managed` permits developer-owned history providers or downstream -service storage; it is not restricted to `HistoryProvider`. Remove the constructor `store=` alias, -including acceptance of `None`, without changing the wire flag or downstream storage controls. +service storage; it is not restricted to `HistoryProvider`. The constructor selects the backend only through +`response_store`; this does not change the wire flag or downstream storage controls. ### Background execution and recovery Treat outer background execution, inner inference-service background execution, and recovery permission as separate controls. Use `background_source="host_managed"` by default. Retain `"inference_service"` for supported regular agents with inference-service history and outer-background passthrough. -Reject hidden provider background under an outer foreground request. +Validate hosting-visible agent and factory defaults against the selected background mode. Outer workflow background must not enable background on constituent agents. Native inner provider background is unsupported. Do not introduce a per-agent workflow-host background switch. Recovery permission neither selects an execution mode nor guarantees exactly-once external side effects. +For regular Responses and Invocations agents, allow an application options hook to select `background`; +the application owns that choice. Ordinary Responses agent runs preserve hook-selected values. When the host +explicitly selects provider background, retain `background=True` for submission and polling even if the hook +returns `False`. Hook-selected background alone does not enable host-managed polling or recovery. Invocations +does not inject an outer background value. + +Do not inject background-specific enforcement middleware or add a Core interception hook. Application agent/chat +middleware owns its changes to background options. Keep validation of caller envelope fields, hook results +other than the agent hook's background choice, and developer defaults before execution. Reject nested caller +`extra_body.background` before the Responses hook. Native workflow option hooks cannot select inner provider +background; make native `WorkflowTurn` reject client-level and run-level middleware. The existing native agent +options middleware still validates transport controls, storage, and unfinished provider responses to preserve +checkpoint ownership; it is not the removed background-specific guard. + ### Supported host settings Expose supported first-class settings and construct SDK options internally, without another framework @@ -134,28 +162,45 @@ or other request-option precedence. ### Approvals and consent -Use session-backed authority for regular-agent approvals rather than a separate public approval-store API. -Native workflow pending authority remains checkpoint-owned. Canonical pending state, conditional consumption, -persistence ordering, custom-store capabilities, and failure semantics require a complete design before -replacing existing approval storage. - -Reject incomplete hosted MCP approval sets with HTTP 400 before streaming, response headers, -`response.created`, or protected dispatch, without consuming pending authority. -A complete batch may mix approvals and rejections. +Use the framework's built-in approval handling and do not add a host approval authority. +Regular agents keep pending approval requests in the same `AgentSession` that the host loads and saves through +the configured session store. Core binds each approval response to a recorded pending request, rebuilds the call +from that request, and consumes it. The host stores only a translation map from the SDK-generated wire item ID to +the original request in that session's state. This map does not authorize execution. Native workflows use +`request_info` and their exact checkpoint; the host translates wire IDs to request IDs in its checkpoint binding. +There is no separate public approval-store API, no conditional-write capability for custom session stores, and no +import of approvals recorded outside session state. + +Do not preflight approval replies before the response starts. The host drops a reply with an unknown wire ID +with a warning, and the framework does not execute a reply that matches no pending request. A turn whose input +contains only dropped replies fails. Partial batches follow the framework: an agent holds the batch until every +request has a reply, and a workflow runs the answered requests. Responses hosts emit requests that are still +pending again under new wire IDs; Invocations workflows keep their request IDs. A batch may mix approvals and rejections. A workflow reply that the runtime rejects before its first superstep releases the +lineage head unchanged; a failure after a superstep starts blocks that lineage. Concurrent replies, forks, and +retries have the same guarantees as any agent run with a shared session or any workflow checkpoint resume. +No path provides exactly-once external side effects. Represent invocation-time OAuth consent as structured consent with incomplete response status. Pair each emitted function call with one terminal result, preserve completed sibling results, and stop later protected calls. Require `store=true` so continuation can retain the required state; do not support ephemeral consent under `store=false`. +Buffer consent and approval output until the session snapshots are saved. Invocation-time consent with +inference-service history is unsupported and fails explicitly; connection-time consent remains separate. +A consent failure after execution starts is a failed response, not a pre-header HTTP 400. + +### Caller options + +Preserve protocol-specific option ordering. Responses translates generation fields before factory resolution +and its hook; Invocations parses and applies its hook before factory resolution. Deep-copy per-run hook input +and replacement mappings without changing developer defaults. Protect hosting and execution controls after +the hook, and reject those controls inside nested transport options. The host split does not authorize +removing existing hooks or applying workflow-only assumptions to regular agents. ### Unresolved boundaries Do not infer a single-target restriction or a multi-target router without a documented Foundry platform contract. Request `model` is not implicitly an agent selector. -Broader option precedence and validation details require separate design. The host split does not -authorize removing existing hooks or applying workflow-only assumptions to regular agents. - Keep current dependency requirements and the steering guard until published-wheel compatibility and concurrent queue-overflow behavior are verified. An upstream merge alone is not release evidence. These decisions do not redesign Core, concurrency, checkpoints, retries, or external exactly-once execution. @@ -165,13 +210,14 @@ These decisions do not redesign Core, concurrency, checkpoints, retries, or exte - Public signatures identify execution models and their state owners. - Shared internals avoid duplicate engines without obscuring public contracts. - Request-aware factories support identity-dependent resources but require deliberate isolation and cleanup. -- The API migration breaks combined-host construction, wrapper hosting, unnamed or zero-argument agent factories, old parser/request names, source literals, and the storage-backend alias. +- Agent and workflow hosts have separate constructors and validation; wrapper hosting and unnamed or zero-argument agent factories are not supported. - Settings, authority, and consent changes require their own implementation and correctness validation; recording a decision does not establish that it has shipped. ## Validation Cover public imports and signatures, invalid construction before SDK side effects, sync/async factory -invocation, failure/cancellation cleanup, and rejection of shared workflow resources. +invocation, cleanup of successfully entered contexts on failure/cancellation, and rejection of shared +workflow resources. Exercise streaming and non-streaming execution, session/checkpoint continuation, pending replies, and model-free process crash/recovery. Validate maintained samples against public exports. Offline checks do not establish live Foundry deployment compatibility. diff --git a/python/packages/foundry_hosting/README.md b/python/packages/foundry_hosting/README.md index f37178cad2f..94d29719bbc 100644 --- a/python/packages/foundry_hosting/README.md +++ b/python/packages/foundry_hosting/README.md @@ -2,10 +2,7 @@ This package provides the integration of Agent Framework agents and workflows with the Foundry Agent Server, which can be hosted on Foundry infrastructure. -Implementation is grouped by protocol: `_responses_agent.py` and `_responses_workflow.py` define the Responses hosts, -and `_responses_shared.py` contains their wire and streaming helpers. Invocations uses the same `_invocations_agent.py`, -`_invocations_workflow.py`, and `_invocations_shared.py` layout. Cross-protocol request contracts, option validation, -and agent/workflow factories live in `_shared.py`. Applications use the public package imports below. +See [ADR 0055](../../../docs/decisions/0055-python-foundry-hosting-contracts.md) for the design rationale. ## Choose the host @@ -13,47 +10,91 @@ Use `ResponsesHostServer` or `InvocationsHostServer` for agents. Use `ResponsesWorkflowHostServer` or `InvocationsWorkflowHostServer` for native workflows. Each workflow host requires a request-aware sync/async factory and an explicit parser. All four hosts remain in `agent-framework-foundry-hosting`, with matching lazy exports -under `agent_framework.foundry`. Invocations and Toolbox retain their experimental boundary; -the split does not promote them. - -**Breaking migration:** replace `ResponsesHostServer(workflow=..., parse_response=...)` with -`ResponsesWorkflowHostServer(workflow=..., parse_input=...)`, and replace -`InvocationsHostServer(workflow=..., parse_request=...)` with -`InvocationsWorkflowHostServer(workflow=..., parse_input=...)`. -There is no combined-host compatibility shim. Agent host construction still uses `agent=`. -All custom parsers now use `parse_input`, including agent parsers on `InvocationsHostServer`. -Rename `HostedResponseRequest` imports and annotations to `ResponsesRequest`, including factories and option hooks. -`InvocationRun` and `WorkflowTurn` are unchanged. Workflow parsers still adapt either typed start input -or a complete batch of pending replies; they are not limited to initial input or raw JSON deserialization. -The old parser keywords and request-type name have no compatibility aliases. - -Agent history/background source literals also change without aliases: - -| Setting | Previous literal | New literal | +under `agent_framework.foundry`. The package is currently in beta, so its public APIs are +experimental. Responses hosting is the target for an initial stable release, subject to +qualification. Invocations hosting and `FoundryToolbox` remain experimental. + +Agent hosts accept `agent=` and an optional `parse_input` (Invocations) or no parser (Responses). +Workflow hosts accept `workflow=` and a required `parse_input`. Agent parsers return `InvocationRun`; +workflow parsers return `WorkflowTurn`, with either typed start input or a complete batch of pending replies. +Factories and option hooks on Responses hosts receive a `ResponsesRequest`. + +**Caller options are opt-in.** By default an agent host forwards only the caller's messages. The agent's own +`default_options` and client defaults apply unchanged. To accept caller generation settings, configure +`prepare_options` on an agent host. Only the mapping that the hook returns, after validation, is forwarded. +Workflow hosts have no option hook: the parser forwards model options explicitly through +`WorkflowTurn(client_kwargs={"options": ...})`. The original request remains available to application code. + +Agent history and background sources use these literals: + +| Setting | Values | Default | | --- | --- | --- | -| `history_source` | `"agent_server"` | `"host_managed"` | -| `history_source` | `"agent"` | `"agent_managed"` | -| `history_source` | `"service"` | `"inference_service"` | -| `background_source` | `"agent_server"` | `"host_managed"` | -| `background_source` | `"provider"` | `"inference_service"` | - -These are terminology changes only. The default remains host-managed history and background execution. -`agent_managed` retains developer-owned provider **or** service storage, not just `HistoryProvider` storage. -`response_store=` and the wire-level `store` flag keep their existing meanings. -The deprecated constructor `store=` backend alias is removed. Replace it with `response_store=`; -even `store=None` is rejected. This does not rename the caller's per-request `store` flag. - -`WorkflowSource` now describes a factory only; passing a built workflow fails at construction, -including local one-shot use. Move graph and executor construction inside the factory; +| `history_source` | `"host_managed"`, `"agent_managed"`, `"inference_service"` | `"host_managed"` | +| `background_source` | `"host_managed"`, `"inference_service"` | `"host_managed"` | + +`agent_managed` covers developer-owned provider **or** service storage, not just `HistoryProvider` storage. +`response_store=` selects the response store backend; the caller's per-request `store` flag is separate. + +`WorkflowSource` is a factory; passing a built workflow fails at construction, +including local one-shot use. Build graphs and executors inside the factory; returning a new wrapper around shared executors does not isolate execution state. Workflow hosts expose checkpoint storage, not agent-session storage or agent history/background -policies. Responses workflows retain `prepare_options`; Invocations workflows use the parser's -explicit workflow kwargs instead. Existing SDK options and agent approval-store APIs are unchanged. +policies. Both workflow hosts take model options only from the parser's explicit `WorkflowTurn` +kwargs. Agent approval persistence follows the +[session-store contract below](#function-approvals). Agent hosts reject `WorkflowAgent`, whether supplied directly or returned by a factory. `checkpoint_store_provider` is available only on workflow hosts. No stored state is converted between native workflow and agent execution models. +## Explicit host configuration + +Configure inference defaults on the agent or client, not on the host. Supported hosting settings are: + +| Setting and default | Applies to | Meaning | +| --- | --- | --- | +| `max_history_items=None` | Responses agents | A positive integer caps prior outer history items, only with `history_source="host_managed"`. `None` leaves the installed SDK default unchanged; it does not mean unlimited. | +| `sse_keep_alive_interval_seconds=None` | All four hosts with framed SSE | A positive integer sets the heartbeat interval in seconds. `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`; an absent or nonpositive environment value disables heartbeats. | +| `shutdown_grace_period_seconds=10` | Both Responses hosts | A positive integer sets shutdown waiting in seconds. The SDK also uses it when DELETE waits for response persistence. It is not a model or request timeout. | +| `enable_background_recovery=False` | Both Responses hosts | A boolean permits eligible background recovery. It does not select background execution or guarantee exactly-once effects. | + +For example, after defining the [request-aware factory below](#agent-instances-and-factories): + +```python +server = ResponsesHostServer( + agent=create_agent, + max_history_items=100, + sse_keep_alive_interval_seconds=15, + shutdown_grace_period_seconds=20, +) +``` + +Explicit integer settings reject booleans and non-integers with `TypeError`, and nonpositive values with +`ValueError`. A positive history cap in another agent history mode emits a warning and is ignored. +Each host accepts only the settings listed for it; any other keyword raises `TypeError` at construction. +The upstream Invocations `graceful_shutdown_timeout` passthrough is a separate setting. + +An explicit positive SSE interval takes precedence over a positive environment value. However, the SDK parses +`SSE_KEEPALIVE_INTERVAL` before applying that override: malformed values, including an empty string, still raise +`ValueError` even with an explicit interval. Heartbeats do not set a request timeout. + +| Host | Continuation state | `prepare_options` | Background recovery | +| --- | --- | --- | --- | +| `ResponsesHostServer` | Agent session plus the selected history source | Sync or async | Eligible inference-service background only | +| `ResponsesWorkflowHostServer` | Exact native workflow checkpoint | Not supported; use `WorkflowTurn` kwargs | Checkpoint-backed outer recovery | +| `InvocationsHostServer` | Agent session; application-owned model history | Sync or async | No provider-background recovery | +| `InvocationsWorkflowHostServer` | Exact native workflow checkpoint | Not supported; use `WorkflowTurn` kwargs | No provider-background recovery | + +These are API support boundaries, not live deployment qualification or a promotion of the experimental +surfaces described above. + +## Responses route prefix + +`ResponsesHostServer` and `ResponsesWorkflowHostServer` pass `prefix` to the +[AgentServer Responses host](https://learn.microsoft.com/python/api/azure-ai-agentserver-responses/azure.ai.agentserver.responses.responsesagentserverhost?view=azure-python), +so clients and reverse proxies must use the same path prefix for its +[protocol endpoints](https://learn.microsoft.com/python/api/overview/azure/ai-agentserver-responses-readme?view=azure-python#protocol-endpoints). + ## Toolbox tool approvals `FoundryToolbox` uses each discovered tool's `_meta.tool_configuration.require_approval` when @@ -84,8 +125,8 @@ general-purpose MCP approval policy. `SupportsAgentRun`, and is invoked once for each request as `factory(request=request)`. Factories must declare a required parameter named `request`, either keyword-only or positional-or-keyword. Additional parameters may have defaults, and the factory may ignore the request. -Zero-argument factories and factories with only optional configuration parameters are rejected at construction; -add a required `request` parameter when migrating them. Responses factories receive a +Zero-argument factories and factories with only optional configuration parameters are rejected at construction. +Responses factories receive a `ResponsesRequest`, with trusted scope, current-turn input helpers, and translated caller options. They run before `prepare_options`; the hook can subsequently replace those options. Invocations factories receive the Starlette `Request` after parsing and option preparation. @@ -113,7 +154,7 @@ outside `AgentSession`. Agent hosts continue agents through their session stores instance fields persistent; state needed by later requests must remain in the supported stores. -### Migrate `WorkflowAgent` hosting +### Host workflows natively `ResponsesHostServer` and `InvocationsHostServer` reject `WorkflowAgent` instances and factory results. Host the workflow natively with a request-aware factory returning a built `Workflow`, not `workflow.as_agent()`. @@ -127,8 +168,8 @@ server = ResponsesWorkflowHostServer(workflow=build_workflow, parse_input=parse_ ``` Build fresh workflows, executors, and agents for every request. Keep the workflow name and executor IDs stable so -later requests can find and restore checkpoints. Existing wrapper history, context providers, and event semantics -are not automatically unwrapped or migrated. +later requests can find and restore checkpoints. Wrapper history, context providers, and event semantics +are not unwrapped. ## Native Responses workflows @@ -194,18 +235,20 @@ fresh factory when its model overrides are already materialized in unchanged defaults and a storing client explicitly defaults to `store=False`. Private provider continuation is rejected before output is paired or committed. -For legacy message-input workflows, `response_input_messages(request)` converts +For message-input workflows, `response_input_messages(request)` converts only the current Responses turn to `list[Message]`. It does not load outer -history or decode pending replies. `agent=workflow.as_agent()` is no longer supported. +history or decode pending replies. Agent hosts reject `workflow.as_agent()`. Wrapper context providers, history, event projection, and request-info translation are -not silently unwrapped. See [Migrate `WorkflowAgent` hosting](#migrate-workflowagent-hosting). +not silently unwrapped. See [Host workflows natively](#host-workflows-natively). -With `resilient_background=True`, the application must also enable the +With `enable_background_recovery=True`, the application must also enable the AgentServer resilient task subsystem. Host-owned checkpoint/output pairs recover emitted output and usage without selecting newer unpaired workflow state. This is not an exactly-once guarantee for external tools: make -side-effecting operations idempotent. Legacy unscoped workflow state is not -read; migration starts a fresh Responses chain. +side-effecting operations idempotent. Unscoped workflow state is not +read; start a fresh Responses chain. + +### Factory resource lifecycle For Responses integrations, use a factory when a provider, tool cache, client or credential carries request identity or needs request-owned cleanup. @@ -215,15 +258,23 @@ not retain the first request's call ID. The integration samples still construct it inside the factory because they also own request-scoped clients, credentials and providers. -Factory agents are entered/exited for each request, including failed entry and -cancellation. `Agent` manages context-managed clients and MCP tools, but it does -not automatically manage every context provider or external credential. The +The host unwinds successfully entered factory-agent contexts when a request +finishes, fails or is cancelled. `Agent` also unwinds client and MCP contexts +it has entered if a later context fails to enter. Allocations made before agent +entry and partial allocations from a failed client `__aenter__` may not be +cleaned up. Calling `agent.close()` before entry does not dispose an unentered +client. + +`Agent` manages context-managed clients and MCP tools, but it does not +automatically manage every context provider or external credential. Exiting an +entered `Agent` can close a caller-supplied context-managed client; ownership +depends on the client's context contract. The host cannot reliably identify a +shared client on first use, so applications must ensure that shared clients +support the agent's entry and exit behavior. The [integration samples](../../samples/04-hosting/foundry-hosted-agents/) explicitly own their SDK transports, credentials and providers: Search is request-owned; Memory binds a fresh provider/project client to the trusted user and current call; custom Cosmos state uses user **and** sandbox namespaces with conditional writes. -Only close resources that the factory creates and owns, never a supplied shared -client or somebody else's credential. ## Responses agent history and storage @@ -273,12 +324,7 @@ add a transient in-memory provider to support function-call loops, but **never e The host owns the provided agent instance and any providers it adds; do not reuse it with another host. A factory creates an independent agent for each request. -The existing `history_source="agent_managed"` still preserves the agent's own provider **or** service storage defaults -on stored requests, including `default_options={"store": True}`. Custom `SupportsAgentRun` -implementations can continue using that mode for stored requests; it is not a forced-provider mode. -The outer storage-backend constructor argument is `response_store=`. The old `store=` backend -argument is rejected, including `None` and calls that also supply `response_store=`. -The backend selector does not set the caller's per-request `store` flag. +Custom `SupportsAgentRun` implementations can use `history_source="agent_managed"` for stored requests. `store=False` returns a one-shot response without **writing** host-managed session, conversation, or approval state; it also disables downstream service storage regardless of the developer's defaults. Unsafe custom agents, external @@ -291,7 +337,8 @@ Outer background work always uses the caller-visible `response.id` for polling; background automatically. `background_source="host_managed"` (default) uses only the outer background worker. `background_source="inference_service"` is a separate opt-in for `history_source="inference_service"` with a storing Responses client. Its private continuation token is saved under the outer ID and never returned to the caller. -Use `ResponsesServerOptions(resilient_background=True)` to permit recovery from a **saved** token; a crash before +Set `enable_background_recovery=True` to permit recovery from a **saved** token; for regular agents it requires +`history_source="inference_service"` and `background_source="inference_service"`. A crash before the token is saved cannot safely restart the inner job. Completed polling output, including local function calls, results, and usage, is saved together with the next token in the private response-ID snapshot before emission. Outer output checkpoints record which saved batches have been emitted, so recovery restores their usage and @@ -303,26 +350,79 @@ Each poll retains the caller's generation options and `background=True`, so a to another background response and saves its next token. A crash after a local tool side effect but before that next token is saved can still repeat the tool on recovery; use idempotent tools or avoid provider background for side-effecting local tools. This mechanism does not provide exactly-once tool execution. -Provider background and steering cannot be combined. -Regular agent runs without this opt-in are not crash-replayable. **Steering is temporarily unavailable:** -`steerable_conversations=True` fails during host construction, before enabling the process-wide TaskManager. -The current AgentServer SDK retains unbounded futures for rejected turns when its steering queue fills. Do not -use a queue-length precheck: another worker can append before it. The guard can be removed only after -[Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233) ships in an -official `azure-ai-agentserver-core` wheel, the minimum dependency and `uv.lock` are updated, and a -concurrent queue-overflow regression proves rejected turns leave no pending futures. No future SDK -version is assumed. Non-steerable agent background polling is unchanged. - -Native CreateResponse generation fields become MAF runtime options (notably `max_output_tokens` -> `max_tokens` and -`parallel_tool_calls` -> `allow_multiple_tool_calls`). Flattened OpenAI `extra_body` fields overlay translated keys -**last**. A sync or async `prepare_options(request: ResponsesRequest, options: dict)` hook can remove or replace -*caller* options before `Agent.run`; removed values fall back to the developer's unchanged agent defaults. Hosting -filters caller platform IDs and private continuation/storage controls from model options and rejects attempts to -reintroduce them through the hook. Nested `extra_body` transport overrides are rejected for both caller -input and developer hooks, because they could override the host's `store=False` after the OpenAI SDK merges -the body. Developer defaults also cannot use this transport channel for host-controlled fields on -explicit history modes or unstored requests. A custom agent cannot accept MAF runtime options: choose -`unsupported_options` as `"ignore"`, `"warn"` (default), or `"error"` for that case. See the +Regular agent runs without this opt-in are not crash-replayable. + +The supported regular-agent combinations for host-selected provider background are: + +| `history_source` | `background_source` | Host-selected provider background | `enable_background_recovery=True` | +| --- | --- | --- | --- | +| `"host_managed"` | `"host_managed"` | No; outer background only | Rejected | +| `"agent_managed"` | `"host_managed"` | No; outer background only | Rejected | +| `"inference_service"` | `"host_managed"` | No; outer background only | Rejected | +| `"inference_service"` | `"inference_service"` | Only when the outer request has `background=true` | Supported for saved private tokens | + +`background_source="inference_service"` with either other history mode is rejected. All background requests +require outer `store=true`. For checkpoint-backed native Responses workflow recovery, use +`enable_background_recovery=True` and enable the AgentServer resilient task subsystem; agent history/background +source settings do not apply. This does not enable provider background on workflow agents. + +The host sets provider `background=True` only for an outer background request with both +`history_source="inference_service"` and `background_source="inference_service"`. In that selected mode, +submission and polling retain `background=True`, even if `prepare_options` returns `background=False`. +On ordinary agent runs, a hook-returned `background` value passes through unchanged. Without a hook-selected +value, inference-service background mode passes `False` for an outer foreground request; other modes omit it. +Invocations agents have no outer background mode and do not add a background option. + +An application that selects background in a Responses or Invocations agent hook owns that choice; it does not +enable host-managed provider polling or recovery. Hosting does not inject middleware to inspect or override +background values set by application agent/chat middleware, and adds no Core interception hook. +Factory and developer defaults still undergo background validation before execution. A nested +`extra_body["background"]` in caller options is rejected before the Responses hook, even when it is `False`. + +Native workflows keep a separate checkpoint contract: inner provider background is unsupported, and parser +options cannot select it. Outer workflow `background=True` does not enable it for constituent agents. +A native `WorkflowTurn` rejects client-level and run-level `middleware`. +Its existing native agent options middleware validates transport controls, storage, and unfinished provider +responses to preserve checkpoint ownership; that multipurpose policy remains in place. + +**Steering is unavailable.** The hosts do not enable AgentServer steerable conversations. + +### Preparing caller options + +Caller model-run options are opt-in. Without `prepare_options`, the host clears the translated option mapping +and does not forward caller generation settings. Agent/client defaults and host-owned storage/background controls +still apply. Application factories can still inspect the original request; this default is not an access restriction. + +Native CreateResponse generation fields are translated to MAF runtime options, notably `max_output_tokens` -> `max_tokens` +and `parallel_tool_calls` -> `allow_multiple_tool_calls`. Precedence, from lowest to highest, is: +native generation fields, flattened unknown generation fields, generation fields inside a literal `extra_body` +mapping, then the `prepare_options` replacement. The OpenAI client normally flattens its `extra_body` into the +wire request; a literal nested mapping is also accepted for generation settings. + +A sync or async `prepare_options(request: ResponsesRequest, options: dict[str, Any])` hook receives a fresh +deep copy of this turn's effective caller options. It must return a replacement mapping with string keys; +the host deep-copies that result again. Mutating the argument without returning a mapping is invalid. +Removing `"model"` lets the developer's unchanged agent/client model default apply, while other caller +settings such as `"temperature"` remain only if the hook returns them. Return an explicit allowlist to limit +caller control, or return the supplied mapping to opt in to all options that pass validation. +This works for streaming and non-streaming requests. + +Responses agent ordering is translation, factory resolution and configuration checks, hook, then protected-option +validation before execution. The factory therefore sees translated options **before** the hook. Invocations +agent ordering is parser, hook, protected-option validation, factory, then default validation before entering +the agent. Do not rely on identical HTTP error timing between protocols. Workflow hosts have no option hook; +a Responses workflow parser can read translated caller options from `request.options` and forward the ones it +accepts through `WorkflowTurn(client_kwargs={"options": ...})`. + +Native request-envelope controls such as identity, storage, and continuation fields are not forwarded as +generation options. Putting those controls inside `extra_body` raises `ValueError` +before the hook; the host does not silently strip them. Agent execution controls can reach the hook for filtering, +but cannot be returned. Hook results also cannot contain host controls, +`extra_body`, or execution controls such as `client_kwargs`, `middleware`, `session`, and tool-invocation kwargs. +Responses native `instructions` and `tools` remain supported; Invocations agent options additionally reject both. +Developer defaults cannot use nested transport controls to bypass storage or background policy. +For a custom agent that cannot accept MAF runtime options, choose +`unsupported_options` as `"ignore"`, `"warn"` (default), or `"error"`. See the [agent history and options samples](../../samples/04-hosting/foundry-hosted-agents/responses/basic/). ### OAuth consent origin allowlist @@ -344,6 +444,48 @@ An omitted allowlist preserves existing behavior and does not restrict the HTTPS the gate, so an empty sequence rejects every consent link. Entries are normalized as origins, so paths and query strings belong on the emitted consent link, not in the configuration. +### Invocation-time OAuth consent + +With `history_source="host_managed"` or `"agent_managed"` and `store=true`, a Foundry Toolbox MCP `tools/call` +that requires OAuth consent produces a structured continuation rather than an opaque error. +The Responses agent response has status `incomplete`, and its output contains: + +- the `function_call` for the protected tool; +- a `function_call_output` for that call, whose text says that OAuth consent is required; +- an `oauth_consent_request` item with `consent_link` and `server_label`. + +The host emits a consent request for one `(consent_link, server_label)` pair only once in a response. Non-streaming +calls return these items in the final response. Streaming calls give the same items. The host buffers them and +publishes them only after it saves the session snapshots. + +Other tool calls in the same run behave as follows: + +- Tool calls that completed before the consent error keep their real results. +- The host does not run later protected Toolbox calls in the same run. Each one gets a + "not executed because another tool requires OAuth consent" output. +- Each `function_call` gets exactly one terminal output. + +To continue, the user opens `consent_link` and gives consent. The client then sends a new request with +`previous_response_id`, and the agent tries the tool again. + +Failure boundaries: + +- The response fails if a consent link is not a safe absolute HTTPS URL or is outside the configured origin + allowlist. The host does not expose that URL. +- Consent continuation requires `store=true`. If the request has `store=false`, the response fails and the host + exposes no consent or approval output. Consent appears during tool execution, after the response has started, so + this is a failed response, not an HTTP 400 rejection. +- Invocation-time consent is unsupported with `history_source="inference_service"`: the response fails explicitly + rather than publishing a continuation that the service-owned history cannot safely resume. +- Consent at connection time (`tools/list`) behaves as before. + +Limits: + +- The host does not cancel sibling tool calls that are already running. +- Only Toolbox tools are blocked. Other tools continue to run. +- The host must activate the request scope, so this behavior needs the hosted runtime. +- Core can log the consent exception. + ## Computer use The Responses host emits native `computer_call` and `computer_call_output` items, including ordered `actions`, @@ -381,9 +523,11 @@ Each logical store is saved as one JSON file whose name is a URL-safe Base64 encoding of the store name. For example: - Responses agent sessions: `YWdlbnRfc2Vzc2lvbnM.json` -- Function approvals: `ZnVuY3Rpb25fYXBwcm92YWxz.json` - Workflow checkpoints: one file per context, encoded from `checkpoints/` +The file-based fallback supports ordinary session and checkpoint persistence, including pending agent +approvals. It coordinates writers within one process only. + > Read more about the Foundry durable state store in the [developer guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-core/docs/state-store-guide.md). ### User isolation @@ -417,7 +561,7 @@ conversation or response ID. | MAF `AgentSession.session_id` / `service_session_id` | Inner agent state and optional downstream service continuation; neither identifies the Foundry sandbox. | | Workflow checkpoint ID | Inner workflow state within a checkpoint context, not a Foundry session ID. | -The default hosted MAF session, checkpoint, and approval stores use a `v2` namespace +The default hosted MAF session and checkpoint stores use a `v2` namespace derived from the hashed platform identity, with `user_isolation=True` and an explicit platform `call_id` on each item operation. Checkpoint context IDs are hashed as well. The same user in two hosted sandboxes cannot read the other's default MAF state. @@ -445,9 +589,9 @@ and file-based storage locally. Responses sessions use the `agent_sessions` logi Invocations sessions use the separate `invocation_sessions` store. Each stored agent turn saves a snapshot under its **own** outer `response.id`. For a named `conversation`, a -separate mutable conversation-head key is also updated. Loaded MAF sessions use PR1's ETag condition for that key: -a competing turn that advanced the head causes a visible conflict rather than a stale overwrite. A superseded -steered turn saves its response snapshot but skips the head update. In `"inference_service"` or `"agent_managed"` history mode, a +separate mutable conversation-head key is also updated. Loaded MAF sessions use that key's loaded revision: +a competing turn that advanced the head causes a visible conflict rather than a stale overwrite. In +`"inference_service"` or `"agent_managed"` history mode, a stored named-conversation turn first claims that head with a conditional write *before* calling the inner agent. Another request that read the old head loses the CAS; one that reads the claim fails before touching the provider. The claim is cleared when the winning turn successfully commits the new head. If a dispatched turn fails or is @@ -457,14 +601,15 @@ The committed head also records its completing outer response ID. If a crash occ outer completion, that response can recover its saved final output without reclaiming or rewriting the head. A different in-flight claim or completing response is not accepted as ownership. When continuing by `previous_response_id`, the prior response is claimed with a conditional write **after** -the input is validated, so an invalid approval response does not consume a usable parent. A second branch +the input is validated, so invalid input does not consume a usable parent. A second branch cannot reuse the same downstream service thread; attempting to fork a named service conversation is also rejected. New hosted keys are created only if absent. Custom store providers must provide equivalent scoped conditional writes for concurrent turns, including the pre-dispatch claim. Local callers can still upsert directly without first loading a session. See the [custom storage provider sample](../../samples/04-hosting/foundry-hosted-agents/responses/custom_storage/) -for an example that uses an in-memory session store locally and Azure Cosmos DB when hosted. +for an example that uses an in-memory session store locally and a user-managed Azure Cosmos DB account when hosted. +Its Cosmos prerequisites apply to that custom provider, not to the default Foundry-managed store. Native Responses refusal parts are stored as text carrying `additional_properties["model_output_kind"] == "refusal"` and emitted as @@ -476,24 +621,25 @@ and user ID. Consumers must use it as a whole and must not parse it or depend on representation. Repeated requests for the same identifier pair restore the saved session. Locally, the platform session ID is used unchanged. -### Invocations agent requests and wire compatibility +### Invocations agent requests and response format By default, `POST /invocations` accepts `{"message": "Hi", "options": {}, "stream": false}`. Applications can supply a sync or async `parse_input(request)` returning a typed `InvocationRun(messages, options, stream)` to accept their own JSON shape and MAF `Message` inputs. A sync or async `prepare_options(request, options)` hook can filter or replace a **copy** of this turn's caller options without changing the agent's `default_options`. It must return a mapping -with string keys. The host rejects reserved platform/session fields, `store`, `extra_body`, and private continuation -fields after the hook; callers cannot select another sandbox or enable downstream service continuation through +with string keys; only the returned, validated options are forwarded. Without a hook, caller model-run options +are suppressed and developer defaults apply. The host rejects reserved platform/session fields, `store`, +`extra_body`, and private continuation fields after the hook; callers cannot select another sandbox or enable downstream service continuation through runtime options. It also rejects agent execution controls: `additional_function_arguments`, `function_invocation_kwargs`, `client_kwargs`, `middleware`, `session`, `tools`, `instructions`, `compaction_strategy`, and `tokenizer`. Trusted tool arguments belong in developer-configured agent defaults or middleware/factories, not in request options or hook output. A hook may strip denied caller fields before validation; allowed generation and provider-specific options remain -available. These restrictions apply to both wire formats and every `unsupported_options` policy. +available. These restrictions apply to streaming and non-streaming requests under every `unsupported_options` policy. When an agent cannot accept runtime options, `unsupported_options="warn"` (default) logs and ignores them; `"ignore"` silently drops them and `"error"` rejects them. For request-scoped factories, unsupported options discovered after streaming starts are reported as an SSE `error` event with `status: 400`. -**New default:** non-streaming success is JSON `{"response": "..."}`. Streaming success is real SSE +Non-streaming success is JSON `{"response": "..."}`. Streaming success is framed SSE `event: delta` with `{"text": "..."}`, followed by `event: done` with the platform sandbox `session_id`. The `done` event is sent only after the final MAF `ResponseStream` is finalized and the session is persisted. Client validation errors return JSON HTTP 400 before streaming where possible. Provider errors are logged and @@ -503,14 +649,6 @@ same-session requests in one process, but a CAS conflict can still follow extern processes; it does not guarantee exactly-once execution. See the [Invocations agent/parser example](../../samples/04-hosting/foundry-hosted-agents/invocations/basic/). -**Deprecated opt-in:** set `InvocationsHostServer(agent, legacy_wire_format=True)` only for existing clients -that must keep the previous plain-text non-streaming response and raw text-chunk streaming format. The host logs -and emits a deprecation warning once on construction. Errors are never returned as successful text: non-stream -failures still use JSON error statuses; a post-start legacy stream failure terminates the stream rather than -injecting unexpected SSE framing. Migrate all opted-in clients to JSON/SSE; remove the compatibility mode only -after those callers have migrated and a separate, deliberate breaking-change decision, never by silently -switching an opted-in deployment. - Both hosts accept `agent_session_store_provider` to select a `StoreProvider[SessionStore]`. Session state must support `AgentSession` serialization. Use `register_state_type()` codecs for custom types; unsupported live objects fail during persistence. Restored sessions preserve @@ -533,15 +671,42 @@ locally. Stored checkpoints are scoped under `checkpoints`. ### Function approvals -`ResponsesHostServer` persists function approvals durably. By default, it uses the -`FoundryFunctionApprovalStore`, backed by Foundry storage when hosted and file-based -storage locally. Stored approvals are scoped under `function_approvals`. +`ResponsesHostServer` uses the framework's built-in approval handling. Core records each pending +`function_approval_request` in the `AgentSession`, and the host saves that session with the configured +`agent_session_store_provider`. When a reply arrives, the host loads the same session and translates each +`mcp_approval_response` to a `function_approval_response`. Core binds the reply to its recorded request, +rebuilds the call from that request, and consumes it. A reply that matches no pending request is not executed. + +The SDK generates the wire ID of each `mcp_approval_request` item. The host keeps a map from that wire ID to +the original request in the same session state. The map only translates IDs; it does not authorize execution. +Custom session stores need only the ordinary `SessionStore` operations. + +- Approvals and rejections may be mixed in one reply. +- A partial reply is allowed. Core holds the batch until every request has a reply, and the response emits the + unanswered requests again under new wire IDs. Reply to those new IDs. +- The host drops a reply with an unknown wire ID and logs a warning. A turn whose input contains only dropped + replies fails. +- Invalid replies are not rejected before the response starts. They produce a failed response instead of an + HTTP 400. +- `store=false` cannot persist or resume pending approvals. +- Concurrent replies, forks, and retries have the same guarantees as any agent run that shares a session. + Approval handling does not provide exactly-once tool side effects. + +Native workflows keep pending replies in their `request_info` requests and checkpoint, not in an agent session. +`ResponsesWorkflowHostServer` maps wire IDs to request IDs in the checkpoint binding. A partial reply runs the +answered requests, and the response emits the unanswered requests again under new wire IDs. A reply that the +workflow runtime rejects before its first superstep, such as an unknown request ID or a mismatched reply type, +fails the turn and leaves the lineage head unchanged, so a valid reply can be retried. A failure after a +superstep starts blocks that lineage. + +Pending approvals live in session state. Approvals recorded elsewhere cannot be resumed; start a fresh +conversation and obtain fresh approvals. ## Native Invocations workflows Host a native workflow with an explicit application parser rather than wrapping it with `workflow.as_agent()`, which agent hosts reject (see -[Migrate `WorkflowAgent` hosting](#migrate-workflowagent-hosting)): +[Host workflows natively](#host-workflows-natively)): ```python from agent_framework_foundry_hosting import CheckpointStoreProvider, InvocationsWorkflowHostServer @@ -564,12 +729,14 @@ validates the entire reply batch against its exact pending checkpoint. A sync/async `build_workflow(request)` factory returns a **built**, fresh `Workflow`, with stable name, executor IDs and graph identity. Build fresh executors, agents, -clients, providers and tools for each request; allocation must not execute tools or -workflow side effects. Both hosted and local workflows require a factory; built instances -are rejected at construction. Factory-owned agents are entered and -closed per turn; applications must explicitly own external credentials/providers -through their resource-managing agent subclasses. Do not configure another -checkpoint store on `WorkflowBuilder` or share mutable graphs between requests. +clients, providers and mutable tools for each request. Allocation must not execute +tools or workflow side effects. Both hosted and local workflows require a factory; built instances +are rejected at construction. Successfully entered factory-agent contexts are +unwound per turn; see [factory resource lifecycle](#factory-resource-lifecycle) +for the pre-entry and client-ownership limits. Applications must explicitly own +external credentials and providers through their resource-managing agent +subclasses. Do not configure another checkpoint store on `WorkflowBuilder` or +share mutable graphs between requests. Native `AgentExecutor` graphs support `Agent` and constrained bare `RawAgent`. For `Agent`, hosting applies options through a request-owned agent middleware @@ -585,10 +752,11 @@ caller JSON cannot assert factory ownership. The workflow uses one fixed Invocations lineage inside the trusted user and platform sandbox scope. Later turns restore the **exact** committed checkpoint, not a workflow name's unrelated latest checkpoint. Graph changes, stale writers, -forged/cross-scope/replayed decisions and duplicate authority fail before -executor/response-handler dispatch. Reply with the complete pending batch: -partial, unallowlisted, or invalid batches do not claim or consume any usable -authority, so the complete valid batch can be retried. Native factories +cross-scope and replayed replies fail before executor/response-handler dispatch. +A partial reply runs the answered requests; the rest stay pending under their +request IDs. Unallowlisted batches are rejected before the turn is claimed. A +reply that the workflow runtime rejects before its first superstep fails the turn +and leaves the head unchanged, so a valid reply can be retried. Native factories and explicit `client_kwargs`/`function_invocation_kwargs` keep their existing purposes; they do not introduce arbitrary entry-state or per-executor run-option APIs. Host-controlled identity, storage and execution controls cannot be supplied @@ -607,16 +775,13 @@ then `done` with the **sandbox** `session_id`. Live output is provisional until is conditionally committed. Snapshot retention is bounded by both event count and encoded bytes; exceeding either limit fails the turn without `done`. A conflict, encoding failure, execution failure or -interruption cannot emit successful completion. Failed claimed turns are blocked -to avoid replaying uncertain effects; start a new sandbox instead of blindly +interruption cannot emit successful completion. Turns that fail after their first +superstep starts are blocked to avoid replaying uncertain effects; start a new sandbox instead of blindly retrying non-idempotent work. Checkpoints do not provide exactly-once tool effects. If a connection drops after commit, retrying the same trusted user/sandbox/call ID replays that response's complete stored snapshot without executing the workflow or replacing its head. A different call ID starts normal turn validation and cannot retrieve that snapshot. -Local callers may pass one built workflow for a one-shot non-pausing run. Any -workflow that can pause for external input must use a request-aware factory so a -fresh graph can restore and consume the durable reply on the next request. After a newer Invocations head is durably committed, the host reclaims completed ancestor response records and checkpoints oldest-first, retaining only current, pending, active or blocked authority. Cleanup failures are logged and retried by @@ -629,9 +794,8 @@ including the verified `agent_session_id` query alternative when the platform does not configure `FOUNDRY_AGENT_SESSION_ID`. Body/header IDs cannot establish sandbox routing or select private checkpoints. The agent path still uses its separate `invocation_sessions` store, existing hooks/defaults and JSON/`delta` SSE -contract. Agent-only `legacy_wire_format=True`, `prepare_options` and -`agent_session_store_provider` are rejected for native workflows rather than -silently reinterpreted. +contract. The workflow host does not accept agent-only settings such as +`prepare_options` or `agent_session_store_provider`. See the [typed Ticket/review example](../../samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md#native-workflow-with-typed-tickets) for the parser, stable graph, checkpoint type allowlist, pending replies, diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py index 08a11a53272..3e072a8c534 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py @@ -20,9 +20,6 @@ ContextScopedStoreProvider, FoundryAgentSessionStore, FoundryCheckpointStore, - FoundryFunctionApprovalStore, - FunctionApprovalStore, - FunctionApprovalStoreProvider, StoreProvider, ) from ._toolbox import FoundryToolbox @@ -38,12 +35,9 @@ "ContextScopedStoreProvider": "._state_store", "FoundryAgentSessionStore": "._state_store", "FoundryCheckpointStore": "._state_store", - "FoundryFunctionApprovalStore": "._state_store", "FoundryRequestScope": "._scope", "FoundryToolbox": "._toolbox", "InvocationRun": "._shared", - "FunctionApprovalStore": "._state_store", - "FunctionApprovalStoreProvider": "._state_store", "InvocationsHostServer": "._invocations_agent", "InvocationsWorkflowHostServer": "._invocations_workflow", "ResponsesHostServer": "._responses_agent", @@ -61,11 +55,8 @@ "ContextScopedStoreProvider", "FoundryAgentSessionStore", "FoundryCheckpointStore", - "FoundryFunctionApprovalStore", "FoundryRequestScope", "FoundryToolbox", - "FunctionApprovalStore", - "FunctionApprovalStoreProvider", "InvocationRun", "InvocationsHostServer", "InvocationsWorkflowHostServer", diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_approvals.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_approvals.py new file mode 100644 index 00000000000..2eb8a844d7f --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_approvals.py @@ -0,0 +1,89 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Translate hosted MCP approval wire IDs to the agent's approval requests.""" + +from __future__ import annotations + +import json +from typing import Any, cast + +from agent_framework import AgentSession, Content + +_APPROVAL_STATE_KEY = "__foundry_agent_approval_requests_v1" +_UNSUPPORTED = "Unsupported stored approval requests; start a fresh conversation." + + +class _InvalidApproval(ValueError): + """Reject an approval reply that names no request in this session.""" + + +def _is_approval_request(content: Content) -> bool: + return ( + content.type == "function_approval_request" + and bool(content.id) + and content.function_call is not None + and content.function_call.type == "function_call" + and bool(content.function_call.call_id) + and bool(content.function_call.name) + ) + + +class SessionApprovalState: + """Keep the wire-ID to approval-request map in the agent's own session state. + + The Responses protocol uses wire IDs minted by the host, so the host must map + a reply back to the original ``function_approval_request``. This class does + no I/O and grants no authority. The host persists the map with the ordinary + session snapshot, and Core's session approval tracking alone binds and + consumes pending requests. A forked or retried session snapshot can therefore + answer the same approval again. + """ + + def __init__(self, session: AgentSession) -> None: + self.session = session + + def _requests(self) -> dict[str, str]: + value = self.session.state.get(_APPROVAL_STATE_KEY) + if value is None: + return {} + if not isinstance(value, dict) or any( + not isinstance(key, str) or not key or not isinstance(item, str) + for key, item in cast(dict[object, object], value).items() + ): + raise RuntimeError(_UNSUPPORTED) + return cast(dict[str, str], value) + + async def save_approval_request(self, approval_request_id: str, request: Content) -> None: + """Record the request emitted under ``approval_request_id``.""" + if not isinstance(approval_request_id, str) or not approval_request_id: + raise ValueError("Approval wire ID must be a non-empty string.") + if not _is_approval_request(request): + raise ValueError("Only complete function approval requests can be recorded.") + requests = self._requests() + if approval_request_id in requests: + raise ValueError("Approval wire ID already exists in this session.") + # JSON strings prevent session codecs from interpreting nested provider + # dictionaries with a registered `type` discriminator as live objects. + requests[approval_request_id] = json.dumps(request.to_dict(), allow_nan=False, separators=(",", ":")) + self.session.state[_APPROVAL_STATE_KEY] = requests + + async def load_approval_request(self, approval_request_id: str) -> Content: + """Return the request recorded under ``approval_request_id``. + + Raises: + _InvalidApproval: If this session has no request with that wire ID. + RuntimeError: If the stored map or entry is malformed. + """ + value = self._requests().get(approval_request_id) + if value is None: + raise _InvalidApproval("Approval request is not in this session; start a fresh approval turn.") + try: + payload: Any = json.loads(value) + if not isinstance(payload, dict): + raise TypeError + content = Content.from_dict(cast(dict[str, Any], payload)) + except (TypeError, ValueError, KeyError) as exc: + raise RuntimeError(_UNSUPPORTED) from exc + if not _is_approval_request(content): + raise RuntimeError(_UNSUPPORTED) + return content diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py index 5a55661ac97..fb4fc81cc36 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py @@ -4,21 +4,75 @@ from __future__ import annotations -from collections.abc import Awaitable, Callable -from typing import Any +import asyncio +import inspect +import json +import logging +import sys +import weakref +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping +from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager +from contextvars import Token +from copy import deepcopy +from typing import Any, cast -from agent_framework import SessionStore +from agent_framework import AgentSession, RawAgent, ResponseStream, SessionStore, SupportsAgentRun +from agent_framework._telemetry import mark_feature_used +from azure.ai.agentserver.core import ( + FoundryAgentRequestContext, + get_request_context, + reset_request_context, + set_request_context, +) +from azure.ai.agentserver.invocations import InvocationAgentServerHost from starlette.requests import Request +from starlette.responses import JSONResponse, Response +from ._feature_usage import FeatureIndex from ._invocations_shared import ( - InvocationOptionsHook, - _InvocationsHostServer, # pyright: ignore[reportPrivateUsage] + InvocationStreamingResponse, + hosted_scope, + invocation_failure, + partition_key, + sse_frame, +) +from ._scope import FoundryRequestScope +from ._shared import ( + AgentSource, + InvocationRun, + is_agent, + resolve_agent, + validate_agent_background_options, + validate_agent_source, + validate_default_transport_options, + validate_request_options, + validate_sse_keep_alive_interval, ) -from ._shared import AgentSource, InvocationRun, UnsupportedOptions -from ._state_store import StoreProvider +from ._state_store import AgentSessionStoreProvider, StoreProvider + +logger = logging.getLogger(__name__) + +InvocationParser = Callable[[Request], InvocationRun | Awaitable[InvocationRun]] +InvocationOptionsHook = Callable[[Request, dict[str, Any]], Mapping[str, Any] | Awaitable[Mapping[str, Any]]] +_AGENT_CONTROLLED_FIELDS = frozenset({ + "additional_function_arguments", + "client_kwargs", + "compaction_strategy", + "function_invocation_kwargs", + "instructions", + "middleware", + "session", + "tokenizer", + "tools", +}) -class InvocationsHostServer(_InvocationsHostServer): + +class _IncompatibleRunOptions(TypeError): + """The agent's run method cannot accept the options returned by prepare_options.""" + + +class InvocationsHostServer(InvocationAgentServerHost): """Host an agent with application-defined Invocations turns.""" def __init__( @@ -27,14 +81,17 @@ def __init__( *, openapi_spec: dict[str, Any] | None = None, agent_session_store_provider: StoreProvider[SessionStore] | None = None, - parse_input: Callable[[Request], InvocationRun | Awaitable[InvocationRun]] | None = None, + parse_input: InvocationParser | None = None, prepare_options: InvocationOptionsHook | None = None, - unsupported_options: UnsupportedOptions = "warn", - legacy_wire_format: bool = False, - **kwargs: Any, + sse_keep_alive_interval_seconds: int | None = None, + **agent_server_kwargs: Any, ) -> None: """Create an agent host with session-backed application turns. + By default the host passes only the caller's message to the agent. The agent's own + `default_options` apply unchanged, and caller-supplied options are not forwarded. + Supply `prepare_options` to choose run options per request. + Args: agent: Agent instance or sync/async factory called once per request. Factories must declare a required parameter named `request`, accepting the Starlette `Request` passed by keyword. @@ -49,25 +106,260 @@ def __init__( agent_session_store_provider: Conversation session storage. Defaults to Foundry storage when hosted and file-backed storage locally. Default stores expire 30 days after the last write. parse_input: Sync or async parser returning `InvocationRun`. Without a parser, accept JSON - `message`, optional `options`, and `stream`. - prepare_options: Sync or async hook receiving the request and a copy of caller generation - options. Return the replacement mapping; hosting validates it before agent dispatch. - unsupported_options: Policy for agents without runtime options: `"warn"`, `"ignore"`, or `"error"`. - legacy_wire_format: Deprecated agent-only plain-text and raw-stream compatibility format. - **kwargs: Additional AgentServer host settings forwarded to the SDK. + `message`, `stream`, and optional `options`. Parsed options reach the agent only through + `prepare_options`. + prepare_options: Sync or async hook receiving the request and a copy of the parsed caller options. + Its returned mapping is passed to `agent.run(options=...)` and merges over the agent's + `default_options`. Hosting rejects session, tool, middleware, and storage fields after the + hook and before resolving the factory. A `background` value is the application's choice. + The agent's `run` must accept `options`; otherwise the request fails with HTTP 400. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval. If omitted, use a positive + `SSE_KEEPALIVE_INTERVAL`, otherwise disable heartbeats. Not a request/model timeout. + **agent_server_kwargs: Additional AgentServer SDK constructor arguments, such as + `asyncapi_spec_json` and `asyncapi_spec_yaml`. Raises: - TypeError: If the agent source or parser/hook is invalid, or native workflow settings are supplied. + TypeError: If the agent source, parser/hook, or heartbeat type is invalid. + ValueError: If the heartbeat interval is nonpositive or an injected agent has unsupported + background or transport defaults. """ - if {"workflow", "checkpoint_store_provider"}.intersection(kwargs): - raise TypeError("Use InvocationsWorkflowHostServer for native workflows and checkpoint storage.") - super().__init__( - agent=agent, - openapi_spec=openapi_spec, - agent_session_store_provider=agent_session_store_provider, - parse_input=parse_input, - prepare_options=prepare_options, - unsupported_options=unsupported_options, - legacy_wire_format=legacy_wire_format, - **kwargs, + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + validate_agent_source(agent) + if parse_input is not None and not callable(parse_input): + raise TypeError("parse_input must be a callable.") + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + if is_agent(agent): + self._validate_agent_options(agent) + super().__init__(openapi_spec=openapi_spec, **agent_server_kwargs) + if sse_keep_alive_interval_seconds is not None: + self.config.sse_keepalive_interval = sse_keep_alive_interval_seconds + self._agent = agent + self._owns_request_agent = not is_agent(agent) + self._parse_input = parse_input + self._prepare_options = prepare_options + self._session_store_provider = ( + AgentSessionStoreProvider(store_name="invocation_sessions") + if agent_session_store_provider is None + else agent_session_store_provider + ) + self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( + weakref.WeakValueDictionary() + ) + self.invoke_handler(self._handle_invoke) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + def _partition_key( + self, + *, + context: FoundryAgentRequestContext | None = None, + scope: FoundryRequestScope | None = None, + ) -> str | tuple[str, str]: + """Get the session partition key for the current request.""" + return partition_key(self.config, get_request_context() if context is None else context, scope) + + @asynccontextmanager + async def _request_agent(self, request: Request) -> AsyncGenerator[SupportsAgentRun]: + agent = await resolve_agent(self._agent, request) + self._validate_agent_options(agent) + async with AsyncExitStack() as resources: + if self._owns_request_agent and isinstance(agent, AbstractAsyncContextManager): + await resources.enter_async_context(agent) + yield agent + + @staticmethod + def _validate_agent_options(agent: SupportsAgentRun) -> None: + validate_agent_background_options(agent, provider_background=False) + if isinstance(agent, RawAgent): + validate_default_transport_options(agent.default_options, allow_agent_store=True) + + @asynccontextmanager + async def _request_session( + self, + partition_key: str | tuple[str, str], + context: FoundryAgentRequestContext, + *, + hosted_scope: FoundryRequestScope | None = None, + ) -> AsyncGenerator[AgentSession]: + session_id = ( + json.dumps(partition_key, separators=(",", ":")) if isinstance(partition_key, tuple) else partition_key + ) + try: + provider = self._session_store_provider + if hosted_scope is not None and not self.config.session_id and type(provider) is AgentSessionStoreProvider: + store = provider._get_scoped_store(context, hosted_scope) # pyright: ignore[reportPrivateUsage] + else: + store = provider.get_store(config=self.config, platform_context=context) + session = await store.get(session_id) + if session is None: + session = AgentSession(session_id=session_id) + except Exception: + logger.exception("Failed to load invocation session") + raise + try: + yield session + finally: + failure = sys.exc_info()[1] + try: + await store.set(session_id, session) + except Exception as exc: + logger.exception("Failed to persist invocation session") + if failure is None: + raise + if isinstance(failure, Exception): + raise RuntimeError( + f"Invocation failed: {str(failure) or type(failure).__name__}; " + f"session persistence also failed: {str(exc) or type(exc).__name__}" + ) from exc + + async def _parse(self, request: Request) -> InvocationRun: + if self._parse_input is not None: + result = self._parse_input(request) + parsed = await result if inspect.isawaitable(result) else result + if not isinstance(parsed, InvocationRun): + raise TypeError("parse_input must return InvocationRun.") + return parsed + + payload = await request.json() + if not isinstance(payload, dict): + raise ValueError("The invocation must be a JSON object.") + body = cast(Mapping[str, Any], payload) + message = body.get("message") + stream = body.get("stream", False) + options = body.get("options", {}) + if not isinstance(message, str): + raise ValueError("message must be a string.") + if not isinstance(stream, bool): + raise ValueError("stream must be a boolean.") + if not isinstance(options, dict): + raise ValueError("options must be an object.") + return InvocationRun( + messages=message if stream else [message], options=cast(Mapping[str, Any], options), stream=stream ) + + async def _options(self, request: Request, parsed: InvocationRun) -> dict[str, Any]: + if self._prepare_options is None: + return {} + result = self._prepare_options(request, deepcopy(dict(parsed.options))) + if inspect.isawaitable(result): + result = await result + if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): + raise TypeError("prepare_options must return a mapping of MAF run options with string keys.") + options = deepcopy(dict(result)) + reserved = _AGENT_CONTROLLED_FIELDS.intersection(options) + if reserved: + raise ValueError(f"Invocations options cannot set agent-controlled fields: {', '.join(sorted(reserved))}.") + validate_request_options(options, allow_background=True) + return options + + @staticmethod + def _agent_kwargs(agent: SupportsAgentRun, options: dict[str, Any]) -> dict[str, Any]: + if not options: + return {} + try: + inspect.signature(agent.run).bind_partial(options=options) + except (TypeError, ValueError): + raise _IncompatibleRunOptions( + "The hosted agent does not accept the run options returned by prepare_options." + ) from None + return {"options": options} + + @staticmethod + async def _close_interrupted_stream(stream: object) -> None: + if isinstance(stream, ResponseStream): + close = getattr(cast(object, stream), "close", None) + if close is None: + logger.warning("The installed core cannot close an interrupted agent stream.") + return + else: + close = getattr(stream, "aclose", None) + if close is not None: + await close() + + async def _handle_invoke(self, request: Request) -> Response: + """Invoke the configured agent with a trusted request.""" + context = get_request_context() + try: + request_scope = hosted_scope(self.config, request, context) if self.config.is_hosted else None + key = partition_key(self.config, context, request_scope) + except RuntimeError as exc: + logger.error("Failed to resolve Invocations session: %s", exc) + return JSONResponse({"error": str(exc)}, status_code=500) + + try: + parsed = await self._parse(request) + options = await self._options(request, parsed) + if is_agent(self._agent): + self._agent_kwargs(self._agent, options) + except (TypeError, ValueError) as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception: + logger.exception("Failed to prepare Invocations request") + return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) + + if parsed.stream: + + async def stream_response() -> AsyncGenerator[str]: + token: Token[FoundryAgentRequestContext] | None = set_request_context(context) + try: + lock = self._session_locks.setdefault(key, asyncio.Lock()) + async with lock, self._request_agent(request) as agent: + run_kwargs = self._agent_kwargs(agent, options) + async with self._request_session(key, context, hosted_scope=request_scope) as session: + stream = agent.run(parsed.messages, session=session, stream=True, **run_kwargs) + completed = False + try: + async for update in stream: + if update.text: + # The SDK may close a suspended generator from another task. + reset_request_context(token) + token = None + yield sse_frame("delta", {"text": update.text}) + token = set_request_context(context) + if isinstance(stream, ResponseStream): + await stream.get_final_response() + completed = True + finally: + if not completed: + if token is None: + token = set_request_context(context) + try: + await self._close_interrupted_stream(stream) + except Exception: + logger.exception("Failed to close interrupted Invocations agent stream") + session_id = key[0] if isinstance(key, tuple) else key + if token is not None: + reset_request_context(token) + token = None + yield sse_frame("done", {"session_id": session_id}) + except _IncompatibleRunOptions as exc: + if token is not None: + reset_request_context(token) + token = None + yield sse_frame("error", {"message": str(exc), "code": "invalid_request", "status": 400}) + except Exception as exc: + logger.exception("Invocations agent stream failed") + message, status, code = invocation_failure(exc) + if token is not None: + reset_request_context(token) + token = None + yield sse_frame("error", {"message": message, "code": code, "status": status}) + finally: + if token is not None: + reset_request_context(token) + + return InvocationStreamingResponse(stream_response()) + + try: + lock = self._session_locks.setdefault(key, asyncio.Lock()) + async with lock, self._request_agent(request) as agent: + run_kwargs = self._agent_kwargs(agent, options) + async with self._request_session(key, context, hosted_scope=request_scope) as session: + response = await agent.run(parsed.messages, session=session, **run_kwargs) + return JSONResponse({"response": response.text}) + except _IncompatibleRunOptions as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception as exc: + logger.exception("Invocations agent request failed") + message, status, _ = invocation_failure(exc) + return JSONResponse({"error": message}, status_code=status) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py index 98f5a5de3d7..4347c42169e 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py @@ -1,248 +1,27 @@ # Copyright (c) Microsoft. All rights reserved. -"""Share Invocations protocol, state, and execution plumbing between agent and workflow hosts.""" +"""Protocol-neutral Invocations helpers used by the agent and workflow hosts.""" from __future__ import annotations -import asyncio -import inspect import json -import logging -import math -import sys -import uuid -import warnings -import weakref -from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping, Sequence -from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager -from contextvars import Token -from copy import deepcopy -from dataclasses import asdict, is_dataclass -from typing import Any, cast +from collections.abc import AsyncGenerator, Mapping +from typing import Any -from agent_framework import ( - AgentResponse, - AgentResponseUpdate, - AgentSession, - ChatResponse, - ChatResponseUpdate, - CheckpointStorage, - ResponseStream, - SessionStore, - SupportsAgentRun, - WorkflowCheckpointException, - WorkflowEvent, -) -from agent_framework._telemetry import mark_feature_used -from anyio import CancelScope -from azure.ai.agentserver.core import ( - FoundryAgentRequestContext, - get_request_context, - reset_request_context, - set_request_context, -) +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext from azure.ai.agentserver.core.storage import FoundryStorageConflictError, FoundryStoragePreconditionError -from azure.ai.agentserver.invocations import InvocationAgentServerHost -from pydantic import BaseModel from starlette.requests import Request -from starlette.responses import JSONResponse, Response, StreamingResponse +from starlette.responses import StreamingResponse from starlette.types import Receive, Scope, Send -from ._feature_usage import FeatureIndex from ._scope import FoundryRequestScope -from ._shared import ( - AgentSource, - InvocationRun, - UnsupportedOptions, - WorkflowResolver, - WorkflowSource, - WorkflowTurn, - is_agent, - resolve_agent, - validate_agent_source, - validate_request_options, - validate_unsupported_options, - workflow_agents, -) -from ._state_store import AgentSessionStoreProvider, CheckpointStoreProvider, ContextScopedStoreProvider, StoreProvider -from ._workflow_state import HostedWorkflowRun, WorkflowBlockedError, WorkflowConflictError -logger = logging.getLogger(__name__) -InvocationParser = Callable[[Request], InvocationRun | WorkflowTurn[Any] | Awaitable[InvocationRun | WorkflowTurn[Any]]] -InvocationOptionsHook = Callable[[Request, dict[str, Any]], Mapping[str, Any] | Awaitable[Mapping[str, Any]]] - -_AGENT_CONTROLLED_FIELDS = frozenset({ - "additional_function_arguments", - "client_kwargs", - "compaction_strategy", - "function_invocation_kwargs", - "instructions", - "middleware", - "session", - "tokenizer", - "tools", -}) -_MAX_WORKFLOW_SNAPSHOT_EVENTS = 4096 -_MAX_WORKFLOW_SNAPSHOT_BYTES = 4 * 1024 * 1024 - - -class _UnsupportedAgentOptions(TypeError): - """The agent cannot accept the caller's run options under the selected policy.""" - - -class _WorkflowOutputError(TypeError): - """Workflow application data cannot be represented as JSON.""" - - -class _WorkflowRequestError(ValueError): - """A validated application request cannot be dispatched to the workflow.""" - - -def _validate_workflow_mapping_keys(value: Any, path: str, active: set[int]) -> None: - if isinstance(value, Mapping): - mapping = cast(Mapping[object, Any], value) - identifier = id(mapping) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") - active.add(identifier) - try: - for index, (key, item) in enumerate(mapping.items()): - if not isinstance(key, str): - raise _WorkflowOutputError( - f"Workflow output at {path} is not JSON-serializable: " - f"mapping keys must be strings, found {type(key).__name__}." - ) - _validate_workflow_mapping_keys(item, f"{path}.value[{index}]", active) - finally: - active.remove(identifier) - elif isinstance(value, (list, tuple)): - sequence = cast(Sequence[Any], value) - identifier = id(sequence) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") - active.add(identifier) - try: - for index, item in enumerate(sequence): - _validate_workflow_mapping_keys(item, f"{path}[{index}]", active) - finally: - active.remove(identifier) - - -def _workflow_serialized_value(value: Any, path: str) -> Any: - if isinstance(value, (AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate)): - return value.to_dict(exclude={"continuation_token"}) - if isinstance(value, BaseModel): - _validate_workflow_mapping_keys(value.model_dump(mode="python"), f"{path}.model", set()) - return value.model_dump(mode="json") - if is_dataclass(value) and not isinstance(value, type): - return asdict(value) - to_dict = getattr(value, "to_dict", None) - if callable(to_dict): - return to_dict() - raise _WorkflowOutputError( - f"Workflow output at {path} is not JSON-serializable: unsupported type {type(value).__name__}." - ) - - -def _workflow_json_value(value: Any, path: str, active: set[int]) -> Any: - if value is None or isinstance(value, (str, int, bool)): - return value - if isinstance(value, float): - if not math.isfinite(value): - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: number must be finite.") - return value - if isinstance(value, Mapping): - mapping = cast(Mapping[object, Any], value) - identifier = id(mapping) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") - active.add(identifier) - try: - result: dict[str, Any] = {} - for index, (key, item) in enumerate(mapping.items()): - if not isinstance(key, str): - raise _WorkflowOutputError( - f"Workflow output at {path} is not JSON-serializable: " - f"mapping keys must be strings, found {type(key).__name__}." - ) - result[key] = _workflow_json_value(item, f"{path}.value[{index}]", active) - return result - finally: - active.remove(identifier) - if isinstance(value, (list, tuple)): - sequence = cast(Sequence[Any], value) - identifier = id(sequence) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") - active.add(identifier) - try: - return [_workflow_json_value(item, f"{path}[{index}]", active) for index, item in enumerate(sequence)] - finally: - active.remove(identifier) - identifier = id(value) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular serialized value.") - active.add(identifier) - try: - serialized = _workflow_serialized_value(value, path) - return _workflow_json_value(serialized, f"{path}.serialized", active) - finally: - active.remove(identifier) - - -def _workflow_output(value: Any) -> Any: - """Normalize typed application data without text coercion or skipped values.""" - try: - normalized = _workflow_json_value(value, "$", set()) - return json.loads(json.dumps(normalized, allow_nan=False, ensure_ascii=False).encode("utf-8")) - except _WorkflowOutputError: - raise - except (TypeError, ValueError, RecursionError) as exc: - raise _WorkflowOutputError(f"Workflow output type {type(value).__name__} is not JSON-serializable.") from exc - - -def _workflow_frame(event: WorkflowEvent[Any]) -> dict[str, Any] | None: - if event.type not in ("output", "request_info"): - return None - frame: dict[str, Any] = {"type": event.type, "data": event.data} - if event.type == "request_info": - frame["request_id"] = event.request_id - frame["source_executor_id"] = event.source_executor_id - else: - frame["executor_id"] = event.executor_id - return cast(dict[str, Any], _workflow_output(frame)) - - -def _workflow_snapshot_frames(snapshot: Mapping[str, Any] | None) -> list[dict[str, Any]]: - if snapshot is None or set(snapshot) != {"output"} or not isinstance(snapshot["output"], list): - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - frames: list[dict[str, Any]] = [] - for item in cast(list[Any], snapshot["output"]): - if not isinstance(item, Mapping): - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - frame = cast(dict[str, Any], _workflow_output(item)) - if frame.get("type") == "output": - if set(frame) != {"type", "data", "executor_id"}: - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - elif frame.get("type") == "request_info": - if set(frame) != {"type", "data", "request_id", "source_executor_id"}: - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - else: - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - frames.append(frame) - return frames - - -def _workflow_frame_size(frame: Mapping[str, Any]) -> int: - return len(json.dumps(frame, ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) - - -def _sse(event: str, data: Mapping[str, Any]) -> str: +def sse_frame(event: str, data: Mapping[str, Any]) -> str: return f"event: {event}\ndata: {json.dumps(data)}\n\n" -def _invocation_failure(exc: Exception) -> tuple[str, int, str]: +def invocation_failure(exc: Exception) -> tuple[str, int, str]: cause: BaseException | None = exc while cause is not None: if isinstance(cause, (FoundryStorageConflictError, FoundryStoragePreconditionError)): @@ -251,30 +30,7 @@ def _invocation_failure(exc: Exception) -> tuple[str, int, str]: return "Agent invocation failed.", 500, "invocation_failed" -def _workflow_failure(exc: Exception) -> tuple[str, int, str]: - if isinstance(exc, _WorkflowOutputError): - return str(exc), 500, "invalid_workflow_output" - cause: BaseException | None = exc - while cause is not None: - if isinstance(cause, WorkflowBlockedError): - return ( - "This workflow turn was interrupted or failed; start a new Foundry sandbox.", - 409, - "workflow_blocked", - ) - if isinstance(cause, WorkflowConflictError): - return str(cause), 409, "workflow_conflict" - cause = cause.__cause__ - if _invocation_failure(exc)[1] == 409: - return "Another request advanced this workflow; reload before retrying.", 409, "workflow_conflict" - if isinstance(exc, _WorkflowRequestError): - return str(exc), 400, "invalid_workflow_turn" - if isinstance(exc, PermissionError): - return "Workflow state is not available in this trusted scope.", 403, "workflow_scope" - return "Workflow invocation failed.", 500, "workflow_invocation_failed" - - -class _InvocationStreamingResponse(StreamingResponse): +class InvocationStreamingResponse(StreamingResponse): def __init__(self, content: AsyncGenerator[str]) -> None: super().__init__( content, @@ -297,560 +53,59 @@ async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: await self._content.aclose() -class _WorkflowStreamingResponse(_InvocationStreamingResponse): - def __init__(self, content: AsyncGenerator[str], *, keep_alive: bool) -> None: - super().__init__(content) - self._keep_alive = keep_alive - - async def stream_response(self, send: Send) -> None: - try: - await super().stream_response(send) - finally: - with CancelScope(shield=True): - if self._keep_alive and self.body_iterator is not self._content: - close = getattr(self.body_iterator, "aclose", None) - if close is None: - raise TypeError("The Invocations SDK stream iterator must support aclose.") - await close() - try: - await self._content.aclose() - finally: - # Without the SDK pump, finish its wrapper in the task that created its context tokens. - async for _ in self.body_iterator: - pass - +def partition_key( + config: AgentConfig, + context: FoundryAgentRequestContext, + scope: FoundryRequestScope | None = None, +) -> str | tuple[str, str]: + """Get the isolation key for the current request. -class _InvocationsHostServer(InvocationAgentServerHost): # pyright: ignore[reportUnusedClass] - """Share protocol, scoped state, and execution plumbing between the public hosts.""" + Hosted keys are a ``(session_id, user_id)`` tuple, so one user cannot reach another user's + state. Locally, the key is the session ID. - def __init__( - self, - agent: AgentSource[Request] | None = None, - *, - workflow: WorkflowSource[Request] | None = None, - openapi_spec: dict[str, Any] | None = None, - agent_session_store_provider: StoreProvider[SessionStore] | None = None, - checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - parse_input: InvocationParser | None = None, - prepare_options: InvocationOptionsHook | None = None, - unsupported_options: UnsupportedOptions = "warn", - legacy_wire_format: bool = False, - **kwargs: Any, - ) -> None: - """Initialize shared protocol plumbing after the public host selects its execution model.""" - if "parse_request" in kwargs: - raise TypeError("parse_request was renamed to parse_input.") - if agent is not None and workflow is not None: - raise TypeError("Provide exactly one of agent or workflow.") - if agent is None and workflow is None: - raise TypeError( - "Provide exactly one of agent or workflow; " - "agent must be an agent instance or a callable that creates one." - ) - if workflow is None: - validate_agent_source(agent) - elif parse_input is None: - raise TypeError("parse_input is required when hosting a native Workflow.") - if parse_input is not None and not callable(parse_input): - raise TypeError("parse_input must be a callable.") - if prepare_options is not None and not callable(prepare_options): - raise TypeError("prepare_options must be a callable.") - if not isinstance(legacy_wire_format, bool): - raise TypeError("legacy_wire_format must be a boolean.") - if workflow is not None: - if legacy_wire_format: - raise ValueError("legacy_wire_format=True is incompatible with native workflows.") - if prepare_options is not None: - raise ValueError("prepare_options is agent-only; use the workflow parser's explicit client_kwargs.") - if agent_session_store_provider is not None: - raise ValueError( - "agent_session_store_provider is agent-only; use checkpoint_store_provider for workflows." - ) - self._workflow_resolver: WorkflowResolver[Request] | None = ( - WorkflowResolver(workflow) if workflow is not None else None - ) - super().__init__(openapi_spec=openapi_spec, **kwargs) - self._agent = agent - self._owns_request_agent = agent is not None and not is_agent(agent) - self._parse_input = parse_input - self._prepare_options = prepare_options - self._unsupported_options = validate_unsupported_options(unsupported_options) - self._legacy_wire_format = legacy_wire_format - self._session_store_provider = ( - AgentSessionStoreProvider(store_name="invocation_sessions") - if agent_session_store_provider is None - else agent_session_store_provider - ) - self._checkpoint_storage_provider = ( - CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider - ) - self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( - weakref.WeakValueDictionary() - ) - if legacy_wire_format: - message = ( - "legacy_wire_format=True is deprecated; migrate Invocations clients to JSON responses " - "and framed SSE events before removing this compatibility mode." - ) - warnings.warn(message, DeprecationWarning, stacklevel=3) - logger.warning("DEPRECATION: %s", message) - self.invoke_handler(self._handle_invoke) - mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) - - def _partition_key( - self, - *, - context: FoundryAgentRequestContext | None = None, - scope: FoundryRequestScope | None = None, - ) -> str | tuple[str, str]: - """Get the partition key for the current request. - - A hosted partition key is a tuple containing the session ID and user ID, - preserving their boundaries. Locally, the key is just the session ID. In the - Foundry hosted environment, the partition key is used to maintain isolation between - different sessions and users, such that one user cannot access another user's sessions. - - Returns: - The partition key for the current request. - - Exceptions: - RuntimeError: If the context doesn't contain the expected IDs. - """ - if context is None: - context = get_request_context() - - if self.config.is_hosted: - if not context.user_id: - raise RuntimeError( - "The hosted environment is missing user_id in the request context. " - "Please ensure that the request is coming from a valid Foundry platform service." - ) - hosted_scope = scope or FoundryRequestScope.from_context(self.config, context) - return hosted_scope.session_id, context.user_id - - if not context.session_id: - raise RuntimeError( - "The request context is missing session_id. Please ensure that the request is a valid request." - ) - - return context.session_id - - def _hosted_scope(self, request: Request, context: FoundryAgentRequestContext) -> FoundryRequestScope: - """Accept an unconfigured session only when the routed Invocations query identifies it.""" + Raises: + RuntimeError: If the context does not contain the expected IDs. + """ + if config.is_hosted: if not context.user_id: - raise RuntimeError("The hosted environment is missing user_id in the request context.") - - routed_session_ids = request.query_params.getlist("agent_session_id") - if len(routed_session_ids) > 1: - raise RuntimeError("Hosted Invocations requires exactly one agent_session_id query parameter.") - routed_session_id = routed_session_ids[0] if routed_session_ids else None - if self.config.session_id: - if routed_session_id is not None and routed_session_id != self.config.session_id: - raise RuntimeError("The request agent_session_id does not match the platform session ID.") - return FoundryRequestScope.from_context(self.config, context) - - if not routed_session_id or routed_session_id != context.session_id: raise RuntimeError( - "Hosted Invocations without FOUNDRY_AGENT_SESSION_ID require an explicit routed " - "agent_session_id query parameter matching the request context." + "The hosted environment is missing user_id in the request context. " + "Please ensure that the request is coming from a valid Foundry platform service." ) - if not context.call_id: - raise RuntimeError("Foundry hosted requests require a trusted user ID and call ID.") - return FoundryRequestScope( - session_id=routed_session_id, - user_id=context.user_id, - call_id=context.call_id, - is_hosted=True, - ) + hosted = scope or FoundryRequestScope.from_context(config, context) + return hosted.session_id, context.user_id - @asynccontextmanager - async def _request_agent(self, request: Request) -> AsyncGenerator[SupportsAgentRun]: - if self._agent is None: - raise RuntimeError("No agent is configured for Invocations.") - agent = await resolve_agent(self._agent, request) - async with AsyncExitStack() as resources: - if self._owns_request_agent and isinstance(agent, AbstractAsyncContextManager): - await resources.enter_async_context(agent) - yield agent - - @asynccontextmanager - async def _request_session( - self, - partition_key: str | tuple[str, str], - context: FoundryAgentRequestContext, - *, - hosted_scope: FoundryRequestScope | None = None, - ) -> AsyncGenerator[AgentSession]: - session_id = ( - json.dumps(partition_key, separators=(",", ":")) if isinstance(partition_key, tuple) else partition_key + if not context.session_id: + raise RuntimeError( + "The request context is missing session_id. Please ensure that the request is a valid request." ) - try: - provider = self._session_store_provider - if hosted_scope is not None and not self.config.session_id and type(provider) is AgentSessionStoreProvider: - store = provider._get_scoped_store(context, hosted_scope) # pyright: ignore[reportPrivateUsage] - else: - store = provider.get_store(config=self.config, platform_context=context) - session = await store.get(session_id) - if session is None: - session = AgentSession(session_id=session_id) - except Exception: - logger.exception("Failed to load invocation session") - raise - try: - yield session - finally: - failure = sys.exc_info()[1] - try: - await store.set(session_id, session) - except Exception as exc: - logger.exception("Failed to persist invocation session") - if failure is None: - raise - if isinstance(failure, Exception): - raise RuntimeError( - f"Invocation failed: {str(failure) or type(failure).__name__}; " - f"session persistence also failed: {str(exc) or type(exc).__name__}" - ) from exc - - async def _parse(self, request: Request) -> InvocationRun | WorkflowTurn[Any]: - if self._parse_input is not None: - result = self._parse_input(request) - parsed = await result if inspect.isawaitable(result) else result - if self._workflow_resolver is not None: - if not isinstance(parsed, WorkflowTurn): - raise TypeError("parse_input must return WorkflowTurn.") - elif not isinstance(parsed, InvocationRun): - raise TypeError("parse_input must return InvocationRun.") - return parsed - - payload = await request.json() - if not isinstance(payload, dict): - raise ValueError("The invocation must be a JSON object.") - body = cast(Mapping[str, Any], payload) - message = body.get("message") - stream = body.get("stream", False) - options = body.get("options", {}) - if not isinstance(message, str): - raise ValueError("message must be a string.") - if not isinstance(stream, bool): - raise ValueError("stream must be a boolean.") - if not isinstance(options, dict): - raise ValueError("options must be an object.") - return InvocationRun( - messages=message if stream else [message], options=cast(Mapping[str, Any], options), stream=stream + return context.session_id + + +def hosted_scope(config: AgentConfig, request: Request, context: FoundryAgentRequestContext) -> FoundryRequestScope: + """Accept an unconfigured session only when the routed Invocations query identifies it.""" + if not context.user_id: + raise RuntimeError("The hosted environment is missing user_id in the request context.") + + routed_session_ids = request.query_params.getlist("agent_session_id") + if len(routed_session_ids) > 1: + raise RuntimeError("Hosted Invocations requires exactly one agent_session_id query parameter.") + routed_session_id = routed_session_ids[0] if routed_session_ids else None + if config.session_id: + if routed_session_id is not None and routed_session_id != config.session_id: + raise RuntimeError("The request agent_session_id does not match the platform session ID.") + return FoundryRequestScope.from_context(config, context) + + if not routed_session_id or routed_session_id != context.session_id: + raise RuntimeError( + "Hosted Invocations without FOUNDRY_AGENT_SESSION_ID require an explicit routed " + "agent_session_id query parameter matching the request context." ) - - async def _options(self, request: Request, parsed: InvocationRun) -> dict[str, Any]: - options = deepcopy(dict(parsed.options)) - if self._prepare_options is not None: - result = self._prepare_options(request, options) - if inspect.isawaitable(result): - result = await result - if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): - raise TypeError("prepare_options must return a mapping of MAF run options with string keys.") - options = deepcopy(dict(result)) - validate_request_options(options) - reserved = _AGENT_CONTROLLED_FIELDS.intersection(options) - if reserved: - raise ValueError(f"Invocations options cannot set agent-controlled fields: {', '.join(sorted(reserved))}.") - return options - - def _agent_kwargs(self, agent: SupportsAgentRun, options: dict[str, Any]) -> dict[str, Any]: - if not options: - return {} - try: - inspect.signature(agent.run).bind_partial(options=options) - except (TypeError, ValueError): - if self._unsupported_options == "error": - raise _UnsupportedAgentOptions("The hosted agent does not accept caller runtime options.") from None - if self._unsupported_options == "warn": - logger.warning("Agent doesn't support runtime options. They will be ignored.") - return {} - return {"options": options} - - @staticmethod - async def _close_interrupted_stream(stream: object) -> None: - if isinstance(stream, ResponseStream): - close = getattr(cast(object, stream), "close", None) - if close is None: - logger.warning("The installed core cannot close an interrupted agent stream.") - return - else: - close = getattr(stream, "aclose", None) - if close is not None: - await close() - - async def _handle_invoke(self, request: Request) -> Response: - """Invoke the configured agent or native workflow with a trusted request.""" - context = get_request_context() - try: - hosted_scope = self._hosted_scope(request, context) if self.config.is_hosted else None - partition_key = self._partition_key(context=context, scope=hosted_scope) - except RuntimeError as exc: - logger.error("Failed to resolve Invocations session: %s", exc) - return JSONResponse({"error": str(exc)}, status_code=500) - - try: - parsed = await self._parse(request) - options = await self._options(request, parsed) if isinstance(parsed, InvocationRun) else {} - except (TypeError, ValueError) as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - except Exception: - logger.exception("Failed to prepare Invocations request") - return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) - - if isinstance(parsed, WorkflowTurn): - scope = hosted_scope or FoundryRequestScope.from_context(self.config, context) - return await self._handle_workflow(request, parsed, scope, context, partition_key) - - if parsed.stream: - if options and self._unsupported_options == "error" and is_agent(self._agent): - try: - self._agent_kwargs(self._agent, options) - except _UnsupportedAgentOptions as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - - async def stream_response() -> AsyncGenerator[str]: - token: Token[FoundryAgentRequestContext] | None = set_request_context(context) - try: - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock, self._request_agent(request) as agent: - run_kwargs = self._agent_kwargs(agent, options) - async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: - stream = agent.run(parsed.messages, session=session, stream=True, **run_kwargs) - completed = False - try: - async for update in stream: - if update.text: - frame = ( - update.text - if self._legacy_wire_format - else _sse("delta", {"text": update.text}) - ) - # The SDK may close a suspended generator from another task. - reset_request_context(token) - token = None - yield frame - token = set_request_context(context) - if isinstance(stream, ResponseStream): - await stream.get_final_response() - completed = True - finally: - if not completed: - if token is None: - token = set_request_context(context) - try: - await self._close_interrupted_stream(stream) - except Exception: - logger.exception("Failed to close interrupted Invocations agent stream") - if not self._legacy_wire_format: - session_id = partition_key[0] if isinstance(partition_key, tuple) else partition_key - if token is not None: - reset_request_context(token) - token = None - yield _sse("done", {"session_id": session_id}) - except _UnsupportedAgentOptions as exc: - if self._legacy_wire_format: - raise - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": str(exc), "code": "unsupported_options", "status": 400}) - except Exception as exc: - logger.exception("Invocations agent stream failed") - if self._legacy_wire_format: - raise - message, status, code = _invocation_failure(exc) - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": message, "code": code, "status": status}) - finally: - if token is not None: - reset_request_context(token) - - return _InvocationStreamingResponse(stream_response()) - - try: - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock, self._request_agent(request) as agent: - run_kwargs = self._agent_kwargs(agent, options) - async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: - response = await agent.run(parsed.messages, session=session, **run_kwargs) - if self._legacy_wire_format: - return Response(content=response.text) - return JSONResponse({"response": response.text}) - except _UnsupportedAgentOptions as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - except Exception as exc: - logger.exception("Invocations agent request failed") - message, status, _ = _invocation_failure(exc) - return JSONResponse({"error": message}, status_code=status) - - async def _handle_workflow( - self, - request: Request, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - context: FoundryAgentRequestContext, - partition_key: str | tuple[str, str], - ) -> Response: - if turn.stream: - - async def stream_response() -> AsyncGenerator[str]: - token: Token[FoundryAgentRequestContext] | None = set_request_context(context) - events = self._workflow_events(request, turn, scope, context, partition_key) - try: - async for frame in events: - wire_frame = _sse(frame["type"], frame) - reset_request_context(token) - token = None - yield wire_frame - token = set_request_context(context) - reset_request_context(token) - token = None - yield _sse("done", {"session_id": scope.session_id}) - except Exception as exc: - logger.exception("Invocations workflow stream failed") - message, status, code = _workflow_failure(exc) - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": message, "code": code, "status": status}) - finally: - if token is None: - token = set_request_context(context) - try: - with CancelScope(shield=True): - await events.aclose() - finally: - reset_request_context(token) - - return _WorkflowStreamingResponse(stream_response(), keep_alive=self.config.sse_keepalive_interval > 0) - - try: - frames = [frame async for frame in self._workflow_events(request, turn, scope, context, partition_key)] - return JSONResponse({"output": frames}) - except Exception as exc: - logger.exception("Invocations workflow request failed") - message, status, _ = _workflow_failure(exc) - return JSONResponse({"error": message}, status_code=status) - - async def _workflow_events( - self, - request: Request, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - context: FoundryAgentRequestContext, - partition_key: str | tuple[str, str], - ) -> AsyncGenerator[dict[str, Any]]: - resolver = self._workflow_resolver - if resolver is None: - raise RuntimeError("No native workflow is configured for Invocations.") - frames: list[dict[str, Any]] = [] - snapshot_bytes = 0 - deferred: list[dict[str, Any]] = [] - defer = False - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock: - workflow = await resolver.resolve(request) - run = await HostedWorkflowRun.prepare( - workflow, - scope=scope, - response_id=context.call_id or uuid.uuid4().hex, - config=self.config, - platform_context=context, - checkpoint_store_provider=self._checkpoint_storage_provider, - lineage_id="invocations", - stored=True, - replay_completed=True, - fresh_factory=True, - reclaim_superseded=True, - ) - if run.has_completed_output: - for frame in _workflow_snapshot_frames(run.snapshot): - yield frame - return - try: - turn = run.validate_turn(turn) - except (TypeError, ValueError) as exc: - raise _WorkflowRequestError(str(exc)) from exc - if ( - run.stored - and turn.input is not None - and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) - ): - try: - self._checkpoint_storage_provider.validate_checkpoint_value(turn.input) - except WorkflowCheckpointException as exc: - raise _WorkflowRequestError( - "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " - 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' - ) from exc - if ( - run.stored - and turn.responses is not None - and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) - ): - try: - for reply in turn.responses.values(): - self._checkpoint_storage_provider.validate_checkpoint_value(reply) - except WorkflowCheckpointException as exc: - raise _WorkflowRequestError( - "Stored native workflow reply uses an application type that is not checkpoint-allowlisted. " - 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' - ) from exc - committed = False - resources = AsyncExitStack() - try: - await run.claim() - try: - entered: set[int] = set() - for agent in workflow_agents(workflow): - if id(agent) not in entered and isinstance(agent, AbstractAsyncContextManager): - entered.add(id(agent)) - await resources.enter_async_context(agent) - events = run.events(turn) - resources.push_async_callback(events.aclose) - async for event in events: - frame = _workflow_frame(event) - if frame is None: - continue - frame_size = _workflow_frame_size(frame) - if ( - len(frames) >= _MAX_WORKFLOW_SNAPSHOT_EVENTS - or snapshot_bytes + frame_size > _MAX_WORKFLOW_SNAPSHOT_BYTES - ): - raise _WorkflowOutputError( - "Workflow output exceeds the bounded Invocations snapshot limit." - ) - snapshot_bytes += frame_size - frames.append(frame) - defer = defer or event.type == "request_info" - if defer: - deferred.append(frame) - else: - yield frame - finally: - # Starlette cancellation must not interrupt core/resource cleanup. - with CancelScope(shield=True): - await resources.__aexit__(*sys.exc_info()) - await run.commit({"output": frames}) - committed = True - finally: - if not committed: - failure = sys.exc_info()[1] - try: - with CancelScope(shield=True): - await run.abort() - except Exception as exc: - logger.exception("Failed to persist interrupted Invocations workflow state") - if failure is None: - raise - if isinstance(failure, Exception): - raise RuntimeError( - "Workflow invocation failed and its interrupted state could not be persisted." - ) from exc - # Pending reply authority is durable, and a reply need not wait on this stream's lock. - for frame in deferred: - yield frame + if not context.call_id: + raise RuntimeError("Foundry hosted requests require a trusted user ID and call ID.") + return FoundryRequestScope( + session_id=routed_session_id, + user_id=context.user_id, + call_id=context.call_id, + is_hosted=True, + ) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py index 3f956209228..15f2b858250 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py @@ -4,28 +4,269 @@ from __future__ import annotations -from collections.abc import Awaitable, Callable -from typing import Any +import asyncio +import inspect +import json +import logging +import math +import sys +import uuid +import weakref +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping, Sequence +from contextlib import AbstractAsyncContextManager, AsyncExitStack +from contextvars import Token +from dataclasses import asdict, is_dataclass +from typing import Any, cast -from agent_framework import CheckpointStorage +from agent_framework import ( + AgentResponse, + AgentResponseUpdate, + ChatResponse, + ChatResponseUpdate, + CheckpointStorage, + WorkflowCheckpointException, + WorkflowEvent, +) +from agent_framework._telemetry import mark_feature_used +from anyio import CancelScope +from azure.ai.agentserver.core import ( + FoundryAgentRequestContext, + get_request_context, + reset_request_context, + set_request_context, +) +from azure.ai.agentserver.invocations import InvocationAgentServerHost +from pydantic import BaseModel from starlette.requests import Request +from starlette.responses import JSONResponse, Response +from starlette.types import Send -from ._invocations_shared import _InvocationsHostServer # pyright: ignore[reportPrivateUsage] -from ._shared import WorkflowSource, WorkflowTurn -from ._state_store import ContextScopedStoreProvider +from ._feature_usage import FeatureIndex +from ._invocations_shared import ( + InvocationStreamingResponse, + hosted_scope, + invocation_failure, + partition_key, + sse_frame, +) +from ._scope import FoundryRequestScope +from ._shared import WorkflowResolver, WorkflowSource, WorkflowTurn, validate_sse_keep_alive_interval, workflow_agents +from ._state_store import CheckpointStoreProvider, ContextScopedStoreProvider +from ._workflow_state import HostedWorkflowRun, WorkflowBlockedError, WorkflowConflictError +logger = logging.getLogger(__name__) -class InvocationsWorkflowHostServer(_InvocationsHostServer): +WorkflowParser = Callable[[Request], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]] + +_MAX_WORKFLOW_SNAPSHOT_EVENTS = 4096 +_MAX_WORKFLOW_SNAPSHOT_BYTES = 4 * 1024 * 1024 + + +class _WorkflowOutputError(TypeError): + """Workflow application data cannot be represented as JSON.""" + + +class _WorkflowRequestError(ValueError): + """A validated application request cannot be dispatched to the workflow.""" + + +def _validate_workflow_mapping_keys(value: Any, path: str, active: set[int]) -> None: + if isinstance(value, Mapping): + mapping = cast(Mapping[object, Any], value) + identifier = id(mapping) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") + active.add(identifier) + try: + for index, (key, item) in enumerate(mapping.items()): + if not isinstance(key, str): + raise _WorkflowOutputError( + f"Workflow output at {path} is not JSON-serializable: " + f"mapping keys must be strings, found {type(key).__name__}." + ) + _validate_workflow_mapping_keys(item, f"{path}.value[{index}]", active) + finally: + active.remove(identifier) + elif isinstance(value, (list, tuple)): + sequence = cast(Sequence[Any], value) + identifier = id(sequence) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") + active.add(identifier) + try: + for index, item in enumerate(sequence): + _validate_workflow_mapping_keys(item, f"{path}[{index}]", active) + finally: + active.remove(identifier) + + +def _workflow_serialized_value(value: Any, path: str) -> Any: + if isinstance(value, (AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate)): + return value.to_dict(exclude={"continuation_token"}) + if isinstance(value, BaseModel): + _validate_workflow_mapping_keys(value.model_dump(mode="python"), f"{path}.model", set()) + return value.model_dump(mode="json") + if is_dataclass(value) and not isinstance(value, type): + return asdict(value) + to_dict = getattr(value, "to_dict", None) + if callable(to_dict): + return to_dict() + raise _WorkflowOutputError( + f"Workflow output at {path} is not JSON-serializable: unsupported type {type(value).__name__}." + ) + + +def _workflow_json_value(value: Any, path: str, active: set[int]) -> Any: + if value is None or isinstance(value, (str, int, bool)): + return value + if isinstance(value, float): + if not math.isfinite(value): + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: number must be finite.") + return value + if isinstance(value, Mapping): + mapping = cast(Mapping[object, Any], value) + identifier = id(mapping) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") + active.add(identifier) + try: + result: dict[str, Any] = {} + for index, (key, item) in enumerate(mapping.items()): + if not isinstance(key, str): + raise _WorkflowOutputError( + f"Workflow output at {path} is not JSON-serializable: " + f"mapping keys must be strings, found {type(key).__name__}." + ) + result[key] = _workflow_json_value(item, f"{path}.value[{index}]", active) + return result + finally: + active.remove(identifier) + if isinstance(value, (list, tuple)): + sequence = cast(Sequence[Any], value) + identifier = id(sequence) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") + active.add(identifier) + try: + return [_workflow_json_value(item, f"{path}[{index}]", active) for index, item in enumerate(sequence)] + finally: + active.remove(identifier) + identifier = id(value) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular serialized value.") + active.add(identifier) + try: + serialized = _workflow_serialized_value(value, path) + return _workflow_json_value(serialized, f"{path}.serialized", active) + finally: + active.remove(identifier) + + +def _workflow_output(value: Any) -> Any: + """Normalize typed application data without text coercion or skipped values.""" + try: + normalized = _workflow_json_value(value, "$", set()) + return json.loads(json.dumps(normalized, allow_nan=False, ensure_ascii=False).encode("utf-8")) + except _WorkflowOutputError: + raise + except (TypeError, ValueError, RecursionError) as exc: + raise _WorkflowOutputError(f"Workflow output type {type(value).__name__} is not JSON-serializable.") from exc + + +def _workflow_frame(event: WorkflowEvent[Any]) -> dict[str, Any] | None: + if event.type not in ("output", "request_info"): + return None + frame: dict[str, Any] = {"type": event.type, "data": event.data} + if event.type == "request_info": + frame["request_id"] = event.request_id + frame["source_executor_id"] = event.source_executor_id + else: + frame["executor_id"] = event.executor_id + return cast(dict[str, Any], _workflow_output(frame)) + + +def _workflow_snapshot_frames(snapshot: Mapping[str, Any] | None) -> list[dict[str, Any]]: + if snapshot is None or set(snapshot) != {"output"} or not isinstance(snapshot["output"], list): + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + frames: list[dict[str, Any]] = [] + for item in cast(list[Any], snapshot["output"]): + if not isinstance(item, Mapping): + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + frame = cast(dict[str, Any], _workflow_output(item)) + if frame.get("type") == "output": + if set(frame) != {"type", "data", "executor_id"}: + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + elif frame.get("type") == "request_info": + if set(frame) != {"type", "data", "request_id", "source_executor_id"}: + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + else: + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + frames.append(frame) + return frames + + +def _workflow_frame_size(frame: Mapping[str, Any]) -> int: + return len(json.dumps(frame, ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) + + +def _workflow_failure(exc: Exception) -> tuple[str, int, str]: + if isinstance(exc, _WorkflowOutputError): + return str(exc), 500, "invalid_workflow_output" + cause: BaseException | None = exc + while cause is not None: + if isinstance(cause, WorkflowBlockedError): + return ( + "This workflow turn was interrupted or failed; start a new Foundry sandbox.", + 409, + "workflow_blocked", + ) + if isinstance(cause, WorkflowConflictError): + return str(cause), 409, "workflow_conflict" + cause = cause.__cause__ + if invocation_failure(exc)[1] == 409: + return "Another request advanced this workflow; reload before retrying.", 409, "workflow_conflict" + if isinstance(exc, _WorkflowRequestError): + return str(exc), 400, "invalid_workflow_turn" + if isinstance(exc, PermissionError): + return "Workflow state is not available in this trusted scope.", 403, "workflow_scope" + return "Workflow invocation failed.", 500, "workflow_invocation_failed" + + +class _WorkflowStreamingResponse(InvocationStreamingResponse): + def __init__(self, content: AsyncGenerator[str], *, keep_alive: bool) -> None: + super().__init__(content) + self._keep_alive = keep_alive + + async def stream_response(self, send: Send) -> None: + try: + await super().stream_response(send) + finally: + with CancelScope(shield=True): + if self._keep_alive and self.body_iterator is not self._content: + close = getattr(self.body_iterator, "aclose", None) + if close is None: + raise TypeError("The Invocations SDK stream iterator must support aclose.") + await close() + try: + await self._content.aclose() + finally: + # Without the SDK pump, finish its wrapper in the task that created its context tokens. + async for _ in self.body_iterator: + pass + + +class InvocationsWorkflowHostServer(InvocationAgentServerHost): """Host a native, checkpoint-owned workflow with application-defined Invocations turns.""" def __init__( self, workflow: WorkflowSource[Request], *, - parse_input: Callable[[Request], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], + parse_input: WorkflowParser, openapi_spec: dict[str, Any] | None = None, checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - **kwargs: Any, + sse_keep_alive_interval_seconds: int | None = None, + **agent_server_kwargs: Any, ) -> None: """Create a native workflow host with explicit parsing and checkpoint ownership. @@ -36,33 +277,230 @@ def __init__( agents; applications own other resource cleanup. Known shared execution resources are rejected. Keyword Args: - parse_input: Sync or async parser returning typed `WorkflowTurn` input or a complete pending - reply batch. The application validates the request and explicitly supplies workflow kwargs. + parse_input: Required sync or async parser returning typed `WorkflowTurn` input or a complete + pending reply batch. The parser validates the request and explicitly supplies any workflow + kwargs; the host never converts the request body on its own. openapi_spec: Application OpenAPI specification forwarded to AgentServer. checkpoint_store_provider: Scoped checkpoint storage. Defaults to `CheckpointStoreProvider`. - Native workflow history and pending replies are checkpoint-owned, not agent-session-owned. - **kwargs: Additional AgentServer host settings forwarded to the SDK. Agent option hooks and - legacy wire format are not supported. Output uses JSON or framed SSE workflow events. + Native workflow history and pending replies are checkpoint-owned. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval. If omitted, use a positive + `SSE_KEEPALIVE_INTERVAL`, otherwise disable heartbeats. Not a request/model timeout. + **agent_server_kwargs: Additional AgentServer SDK constructor arguments, such as + `asyncapi_spec_json` and `asyncapi_spec_yaml`. Output uses JSON or framed SSE workflow events. Raises: - TypeError: If the workflow is not a request-aware factory, the parser is invalid, or an agent-only - setting is supplied. + TypeError: If the workflow is not a request-aware factory, the parser is not callable, or the + heartbeat type is invalid. + ValueError: If the heartbeat interval is nonpositive. """ - forbidden = { - "agent", - "agent_session_store_provider", - "prepare_options", - "unsupported_options", - "legacy_wire_format", - }.intersection(kwargs) - if forbidden: - raise TypeError(f"Unsupported native workflow host settings: {', '.join(sorted(forbidden))}.") if not callable(parse_input): - raise TypeError("parse_input must be a callable.") - super().__init__( - workflow=workflow, - parse_input=parse_input, - openapi_spec=openapi_spec, - checkpoint_store_provider=checkpoint_store_provider, - **kwargs, + raise TypeError("parse_input must be a callable returning WorkflowTurn.") + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + self._workflow_resolver: WorkflowResolver[Request] = WorkflowResolver(workflow) + super().__init__(openapi_spec=openapi_spec, **agent_server_kwargs) + if sse_keep_alive_interval_seconds is not None: + self.config.sse_keepalive_interval = sse_keep_alive_interval_seconds + self._parse_input = parse_input + self._checkpoint_storage_provider = ( + CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider + ) + self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( + weakref.WeakValueDictionary() ) + self.invoke_handler(self._handle_invoke) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + async def _parse(self, request: Request) -> WorkflowTurn[Any]: + result = self._parse_input(request) + parsed = await result if inspect.isawaitable(result) else result + if not isinstance(parsed, WorkflowTurn): + raise TypeError("parse_input must return WorkflowTurn.") + return parsed + + async def _handle_invoke(self, request: Request) -> Response: + """Run the native workflow for one trusted request.""" + context = get_request_context() + try: + scope = hosted_scope(self.config, request, context) if self.config.is_hosted else None + key = partition_key(self.config, context, scope) + except RuntimeError as exc: + logger.error("Failed to resolve Invocations session: %s", exc) + return JSONResponse({"error": str(exc)}, status_code=500) + + try: + turn = await self._parse(request) + except (TypeError, ValueError) as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception: + logger.exception("Failed to prepare Invocations request") + return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) + + return await self._handle_workflow( + request, turn, scope or FoundryRequestScope.from_context(self.config, context), context, key + ) + + async def _handle_workflow( + self, + request: Request, + turn: WorkflowTurn[Any], + scope: FoundryRequestScope, + context: FoundryAgentRequestContext, + partition_key: str | tuple[str, str], + ) -> Response: + if turn.stream: + + async def stream_response() -> AsyncGenerator[str]: + token: Token[FoundryAgentRequestContext] | None = set_request_context(context) + events = self._workflow_events(request, turn, scope, context, partition_key) + try: + async for frame in events: + wire_frame = sse_frame(frame["type"], frame) + reset_request_context(token) + token = None + yield wire_frame + token = set_request_context(context) + reset_request_context(token) + token = None + yield sse_frame("done", {"session_id": scope.session_id}) + except Exception as exc: + logger.exception("Invocations workflow stream failed") + message, status, code = _workflow_failure(exc) + if token is not None: + reset_request_context(token) + token = None + yield sse_frame("error", {"message": message, "code": code, "status": status}) + finally: + if token is None: + token = set_request_context(context) + try: + with CancelScope(shield=True): + await events.aclose() + finally: + reset_request_context(token) + + return _WorkflowStreamingResponse(stream_response(), keep_alive=self.config.sse_keepalive_interval > 0) + + try: + frames = [frame async for frame in self._workflow_events(request, turn, scope, context, partition_key)] + return JSONResponse({"output": frames}) + except Exception as exc: + logger.exception("Invocations workflow request failed") + message, status, _ = _workflow_failure(exc) + return JSONResponse({"error": message}, status_code=status) + + async def _workflow_events( + self, + request: Request, + turn: WorkflowTurn[Any], + scope: FoundryRequestScope, + context: FoundryAgentRequestContext, + partition_key: str | tuple[str, str], + ) -> AsyncGenerator[dict[str, Any]]: + resolver = self._workflow_resolver + frames: list[dict[str, Any]] = [] + snapshot_bytes = 0 + deferred: list[dict[str, Any]] = [] + defer = False + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock: + workflow = await resolver.resolve(request) + run = await HostedWorkflowRun.prepare( + workflow, + scope=scope, + response_id=context.call_id or uuid.uuid4().hex, + config=self.config, + platform_context=context, + checkpoint_store_provider=self._checkpoint_storage_provider, + lineage_id="invocations", + stored=True, + replay_completed=True, + fresh_factory=True, + reclaim_superseded=True, + ) + if run.has_completed_output: + for frame in _workflow_snapshot_frames(run.snapshot): + yield frame + return + try: + turn = run.validate_turn(turn) + except (TypeError, ValueError) as exc: + raise _WorkflowRequestError(str(exc)) from exc + if ( + run.stored + and turn.input is not None + and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) + ): + try: + self._checkpoint_storage_provider.validate_checkpoint_value(turn.input) + except WorkflowCheckpointException as exc: + raise _WorkflowRequestError( + "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " + 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' + ) from exc + if ( + run.stored + and turn.responses is not None + and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) + ): + try: + for reply in turn.responses.values(): + self._checkpoint_storage_provider.validate_checkpoint_value(reply) + except WorkflowCheckpointException as exc: + raise _WorkflowRequestError( + "Stored native workflow reply uses an application type that is not checkpoint-allowlisted. " + 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' + ) from exc + committed = False + resources = AsyncExitStack() + try: + await run.claim() + try: + entered: set[int] = set() + for agent in workflow_agents(workflow): + if id(agent) not in entered and isinstance(agent, AbstractAsyncContextManager): + entered.add(id(agent)) + await resources.enter_async_context(agent) + events = run.events(turn) + resources.push_async_callback(events.aclose) + async for event in events: + frame = _workflow_frame(event) + if frame is None: + continue + frame_size = _workflow_frame_size(frame) + if ( + len(frames) >= _MAX_WORKFLOW_SNAPSHOT_EVENTS + or snapshot_bytes + frame_size > _MAX_WORKFLOW_SNAPSHOT_BYTES + ): + raise _WorkflowOutputError( + "Workflow output exceeds the bounded Invocations snapshot limit." + ) + snapshot_bytes += frame_size + frames.append(frame) + defer = defer or event.type == "request_info" + if defer: + deferred.append(frame) + else: + yield frame + finally: + # Starlette cancellation must not interrupt core/resource cleanup. + with CancelScope(shield=True): + await resources.__aexit__(*sys.exc_info()) + await run.commit({"output": frames}) + committed = True + finally: + if not committed: + failure = sys.exc_info()[1] + try: + with CancelScope(shield=True): + await run.abort() + except Exception as exc: + logger.exception("Failed to persist interrupted Invocations workflow state") + if failure is None: + raise + if isinstance(failure, Exception): + raise RuntimeError( + "Workflow invocation failed and its interrupted state could not be persisted." + ) from exc + # Pending reply authority is durable, and a reply need not wait on this stream's lock. + for frame in deferred: + yield frame diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py index 1be4e3109b3..f012e8b8714 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py @@ -5,6 +5,7 @@ from __future__ import annotations import asyncio +import inspect import logging from collections.abc import ( AsyncGenerator, @@ -13,9 +14,9 @@ Mapping, Sequence, ) -from contextlib import AbstractAsyncContextManager, AsyncExitStack, aclosing, suppress -from copy import copy -from dataclasses import dataclass +from contextlib import AbstractAsyncContextManager, AsyncExitStack, aclosing +from copy import copy, deepcopy +from dataclasses import dataclass, field from typing import Any, Literal, cast from agent_framework import ( @@ -51,6 +52,7 @@ ) from azure.ai.agentserver.responses.streaming._checkpoint import ResponseCheckpointEvent +from ._approvals import SessionApprovalState from ._feature_usage import FeatureIndex from ._responses_shared import ( _HOSTED_CONVERSATION_CLAIM_KEY, # pyright: ignore[reportPrivateUsage] @@ -61,6 +63,7 @@ _HOSTED_SERVICE_CHILD_KEY, # pyright: ignore[reportPrivateUsage] _HOSTED_SOURCE_CONVERSATION_KEY, # pyright: ignore[reportPrivateUsage] _PUBLIC_STREAM_FAILURE_MESSAGE, # pyright: ignore[reportPrivateUsage] + _activate_toolbox_consent_scope, # pyright: ignore[reportPrivateUsage] _agent_response_updates, # pyright: ignore[reportPrivateUsage] _await_before_signal, # pyright: ignore[reportPrivateUsage] _create_response_event_stream, # pyright: ignore[reportPrivateUsage] @@ -75,7 +78,6 @@ _reject_busy_conversation, # pyright: ignore[reportPrivateUsage] _set_agent_session, # pyright: ignore[reportPrivateUsage] _SignalledIterator, # pyright: ignore[reportPrivateUsage] - _validate_response_server_options, # pyright: ignore[reportPrivateUsage] consent_url_from_error, ) from ._scope import FoundryRequestScope @@ -83,20 +85,18 @@ AgentSource, OptionsHook, ResponsesRequest, - UnsupportedOptions, + build_agent_response_server_options, is_agent, prepare_response_options, resolve_agent, response_run_options, + validate_agent_background_options, validate_agent_source, validate_default_transport_options, validate_request_options, - validate_unsupported_options, ) from ._state_store import ( AgentSessionStoreProvider, - FunctionApprovalStore, - FunctionApprovalStoreProvider, StoreProvider, ) @@ -127,6 +127,26 @@ class _AgentConfiguration: hosted_history: bool +@dataclass +class _DeferredOutput: + snapshot: dict[str, Any] | None = None + events: list[ResponseStreamEvent | ResponseCheckpointEvent] = field( + default_factory=lambda: list[ResponseStreamEvent | ResponseCheckpointEvent]() + ) + + def defer(self, event: ResponseStreamEvent | ResponseCheckpointEvent) -> bool: + if self.snapshot is None: + return False + self.events.append(event) + return True + + +def _validate_agent_defaults(agent: SupportsAgentRun, *, provider_background: bool, allow_agent_store: bool) -> None: + validate_agent_background_options(agent, provider_background=provider_background) + if isinstance(agent, RawAgent): + validate_default_transport_options(agent.default_options, allow_agent_store=allow_agent_store) + + def _validate_agent_configuration( agent: SupportsAgentRun, history_source: _HistorySource, @@ -137,7 +157,7 @@ def _validate_agent_configuration( resilient_background = bool(options and options.resilient_background) if resilient_background and background_source != "inference_service": raise RuntimeError( - "resilient_background=True requires background_source='inference_service' for regular agents. " + "enable_background_recovery=True requires background_source='inference_service' for regular agents. " "Use ResponsesWorkflowHostServer for checkpoint-based workflow recovery." ) if background_source == "inference_service" and ( @@ -224,15 +244,16 @@ def __init__( agent: AgentSource[ResponsesRequest], *, prefix: str = "", - options: ResponsesServerOptions | None = None, response_store: ResponseProviderProtocol | None = None, agent_session_store_provider: StoreProvider[SessionStore] | None = None, - function_approval_store_provider: StoreProvider[FunctionApprovalStore] | None = None, allowed_oauth_consent_origins: Sequence[str] | None = None, history_source: Literal["host_managed", "agent_managed", "inference_service"] = "host_managed", background_source: Literal["host_managed", "inference_service"] = "host_managed", + max_history_items: int | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + enable_background_recovery: bool = False, prepare_options: OptionsHook | None = None, - unsupported_options: UnsupportedOptions = "warn", **kwargs: Any, ) -> None: """Initialize a ResponsesHostServer. @@ -247,12 +268,10 @@ def __init__( `WorkflowAgent` instances and factory results are rejected. Use `ResponsesWorkflowHostServer` with a native workflow factory instead. prefix: The URL prefix for the server. - options: Optional server options. response_store: Optional response store for caller-facing persistence and history. agent_session_store_provider: Optional provider for MAF agent session storage. - If not provided, a default `AgentSessionStoreProvider` will be used. - function_approval_store_provider: Optional provider for function approval storage. - If not provided, a default `FunctionApprovalStoreProvider` will be used. + If not provided, a default `AgentSessionStoreProvider` will be used. MCP approval + continuations use the same saved `AgentSession`. allowed_oauth_consent_origins: Optional exact HTTPS origins allowed for OAuth consent links. When omitted, hosting retains its existing safe-HTTPS validation without restricting the destination origin. When provided, every link must match an entry; an empty sequence rejects @@ -275,73 +294,76 @@ def __init__( the host-persisted MAF `AgentSession`. With `default_options={"store": True}`, the downstream service retains history instead. - For example, after a stored response to "My name is Ada", a stored follow-up - "What is my name?" sends both turns to the model with `"host_managed"`, but - sends only the follow-up with `"inference_service"` (plus a private service continuation). - With `"agent_managed"`, the developer chooses either of those agent-owned history sources. background_source: `"host_managed"` (default) runs background work in the Responses host without invoking provider-native background APIs; `"inference_service"` opts a storing, resumable client into provider background polling and requires `history_source="inference_service"`. Each poll retains the caller's run options and `background=True`; local tools must be idempotent because a crash before the next private token is saved can replay them. - prepare_options: Developer hook to remove or replace caller model options for an agent. - unsupported_options: `"warn"` (default), `"error"`, or `"ignore"` when a custom agent - cannot accept runtime model options. - **kwargs: Additional keyword arguments. + max_history_items: Positive outer history limit for host-managed history. `None` leaves the + installed SDK default unchanged; it does not mean unlimited. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval. If omitted, the SDK + uses a positive `SSE_KEEPALIVE_INTERVAL` environment value, or disables heartbeats. + shutdown_grace_period_seconds: Positive shutdown wait. The SDK also uses this duration when + DELETE waits for response persistence. Defaults to 10 seconds. + enable_background_recovery: Allow eligible provider-background work to recover after shutdown. + Defaults to `False`; it does not enable background execution or guarantee exactly-once effects. + prepare_options: Optional sync or async hook selecting run options after factory resolution. + Without a hook, only the caller's input messages reach the agent and the agent's own + `default_options` apply unchanged. With a hook, only its returned, validated mapping is + forwarded; an agent whose `run` cannot accept `options` fails the request. + A returned `background` value is application-owned on ordinary runs. Host-selected provider + background submission and polling retain `background=True`. + **kwargs: Additional AgentServer SDK constructor arguments. Note: The *caller* controls outer persistence with `POST /responses` `store=True/False`. `store=False` writes no host-managed session or approval state and forces supported inner clients not to store, regardless of `history_source`. The constructor's `response_store=` argument selects the outer response-store backend. - The former constructor `store=` alias is rejected. Outer `response.id` is the polling/continuation handle; + Outer `response.id` is the polling/continuation handle; any inner `service_session_id` is private and never replaces it. - 1. With `history_source="host_managed"`, - the agent must not have a load-enabled history provider: the host supplies the transcript. - 2. Context providers must not keep required state only on their Python instances, + 1. Context providers must not keep required state only on their Python instances, because the hosting environment may get deactivated between requests. Provider state carried by `AgentSession`, including `InMemoryHistoryProvider` messages in `history_source="agent_managed"` mode, is persisted by the configured session store. - 3. The server owns the supplied agent instance and may add hosting-specific providers. + 2. The server owns the supplied agent instance and may add hosting-specific providers. Do not reuse the same agent with another host or invoke it directly after construction. An agent returned by a callable belongs to that request. - 4. `resilient_background=True` supports regular agents configured with + 3. `enable_background_recovery=True` supports regular agents configured with `history_source="inference_service", background_source="inference_service"`. For provider background, only a saved private continuation token can be polled after a crash; a crash before the token is saved cannot be replayed safely. Other regular-agent runs are not crash-recoverable. - 5. Steering is temporarily unavailable for all agents. `steerable_conversations=True` fails - at construction, before starting a host or enabling the process-wide TaskManager. The current - AgentServer SDK retains futures for rejected turns after the steering queue fills. - 6. A stored, named `"inference_service"` or `"agent_managed"` conversation is claimed in the scoped + 4. A stored, named `"inference_service"` or `"agent_managed"` conversation is claimed in the scoped session store before the agent runs. Conditional writes prevent a concurrent turn from mutating the same provider thread. If the turn fails or is cancelled, start a new conversation instead of reusing a potentially changed downstream thread. Raises: - TypeError: If the agent is a `WorkflowAgent`, or native workflow settings are supplied. - ValueError: If the history, background, or unsupported-options policy is invalid. - RuntimeError: If the agent configuration conflicts with the selected history source, - `resilient_background=True` is requested for a regular agent without provider background, or - `steerable_conversations=True` is requested while steering is unavailable. + TypeError: If the agent is a `WorkflowAgent` or `prepare_options` is not callable. + ValueError: If the history or background settings are invalid. + RuntimeError: If the agent configuration conflicts with the selected history or background source. """ - if "store" in kwargs: - raise TypeError("The constructor store= alias was removed; use response_store= instead.") - _validate_response_server_options(options) - if history_source not in ("host_managed", "agent_managed", "inference_service"): - raise ValueError("history_source must be 'host_managed', 'agent_managed', or 'inference_service'.") - if background_source not in ("host_managed", "inference_service"): - raise ValueError("background_source must be 'host_managed' or 'inference_service'.") - if background_source == "inference_service" and history_source != "inference_service": - raise ValueError("Provider background requires history_source='inference_service'.") - if background_source == "inference_service" and options and options.steerable_conversations: - raise ValueError("Provider background and steerable_conversations cannot be combined.") - if {"workflow", "parse_input", "parse_response", "checkpoint_store_provider"}.intersection(kwargs): - raise TypeError("Use ResponsesWorkflowHostServer for native workflows and checkpoint storage.") validate_agent_source(agent) + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + options = build_agent_response_server_options( + history_source=history_source, + background_source=background_source, + max_history_items=max_history_items, + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + enable_background_recovery=enable_background_recovery, + ) resolved_agent = agent if is_agent(agent) else None + if resolved_agent is not None: + _validate_agent_defaults( + resolved_agent, + provider_background=background_source == "inference_service", + allow_agent_store=history_source == "agent_managed", + ) configuration = ( _validate_agent_configuration(resolved_agent, history_source, options, background_source=background_source) if resolved_agent is not None @@ -356,23 +378,17 @@ def __init__( store=response_store, **kwargs, ) - if options and options.steerable_conversations: - from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled - - set_resilient_tasks_enabled(True) - self._agent_source = agent self._agent = resolved_agent self._configuration = configuration self._history_source: _HistorySource = history_source self._background_source: Literal["host_managed", "inference_service"] = background_source self._prepare_options = prepare_options - self._unsupported_options = validate_unsupported_options(unsupported_options) self._host_options = options self._uses_agent_server_history = ( configuration.agent_server_history if configuration is not None else history_source == "host_managed" ) - self._resilient_background = bool(options and options.resilient_background) + self._resilient_background = enable_background_recovery if resolved_agent is not None and configuration is not None: _initialize_agent_history(resolved_agent, configuration) @@ -380,11 +396,6 @@ def __init__( self._session_storage_provider = ( AgentSessionStoreProvider() if agent_session_store_provider is None else agent_session_store_provider ) - self._function_approval_storage_provider = ( - FunctionApprovalStoreProvider() - if function_approval_store_provider is None - else function_approval_store_provider - ) # Lazy agent lifecycle: the agent (and any MCP tools it owns) is entered on # the first request rather than at server startup, so that authentication @@ -456,6 +467,14 @@ async def _handle_response( raise _PublicResponseFailure(exc) from None agent = await resolve_agent(self._agent_source, hosted_request) try: + provider_background = ( + self._background_source == "inference_service" and request.get("background") is True + ) + _validate_agent_defaults( + agent, + provider_background=provider_background, + allow_agent_store=self._history_source == "agent_managed" and request.get("store") is not False, + ) configuration = self._configuration or _validate_agent_configuration( agent, self._history_source, self._host_options, background_source=self._background_source ) @@ -465,7 +484,20 @@ async def _handle_response( _initialize_agent_history(agent, configuration) await prepare_response_options(hosted_request, self._prepare_options) try: - validate_request_options(hosted_request.options) + validate_request_options(hosted_request.options, allow_background=True) + run_method: Any = agent.run + if hosted_request.options and not isinstance(cast(object, agent), RawAgent): + try: + inspect.signature(run_method).bind_partial(options=hosted_request.options) + except TypeError: + raise TypeError( + "The hosted agent does not accept the run options returned by prepare_options." + ) from None + _validate_agent_defaults( + agent, + provider_background=provider_background, + allow_agent_store=self._history_source == "agent_managed" and request.get("store") is not False, + ) except (TypeError, ValueError) as exc: raise _PublicResponseFailure(exc) from None except Exception as exc: @@ -634,6 +666,7 @@ async def _handle_prepared_response( return tracker = _OutputItemTracker(response_event_stream, self._allowed_oauth_consent_origins) + deferred = _DeferredOutput() try: inner = self._handle_inner_agent( request, @@ -644,14 +677,12 @@ async def _handle_prepared_response( agent, configuration, hosted_request, + deferred, ) - try: + async with aclosing(inner): async for event in inner: yield event - except BaseException: - await inner.aclose() - raise if cancellation_signal.is_set() and context.client_cancelled: # A cancelled run drains the inner generator without raising @@ -684,14 +715,16 @@ async def _handle_prepared_response( else: yield response_event_stream.emit_completed(usage=tracker.usage) except Exception as ex: - for event in tracker.close(): - yield event - + failure_tracker: _OutputItemTracker | None = tracker + if deferred.snapshot is not None: + response_event_stream.response.clear() + response_event_stream.response.update(deferred.snapshot) + failure_tracker = None for event in self._emit_failure( context.response_id, "response production", response_event_stream, - tracker, + failure_tracker, ex, ): yield event @@ -700,8 +733,9 @@ async def _load_request_messages( self, context: ResponseContext, *, - approval_storage: FunctionApprovalStore | None, + approval_storage: SessionApprovalState | None, configuration: _AgentConfiguration | None = None, + approval_ready: asyncio.Future[SessionApprovalState] | None = None, ) -> list[Message]: """Load the request's input and prior history concurrently, assembled for the run. @@ -722,13 +756,23 @@ async def _load_request_messages( async def _load_input() -> list[Message]: input_items = await context.get_input_items() - return await _items_to_messages(input_items, approval_storage=approval_storage) + resolved_storage = await approval_ready if approval_ready is not None else approval_storage + messages = await _items_to_messages(input_items, approval_storage=resolved_storage) + if ( + input_items + and not messages + and all(cast(Mapping[str, Any], item).get("type") == "mcp_approval_response" for item in input_items) + ): + # Do not run the model, or consume a single-use parent, when every reply was dropped. + raise _PublicResponseFailure(ValueError("No approval reply matches a request in this session.")) + return messages async def _load_history() -> list[Message]: if not uses_agent_server_history: return [] history = await context.get_history() - return await _output_items_to_messages(history, approval_storage=approval_storage) + resolved_storage = await approval_ready if approval_ready is not None else approval_storage + return await _output_items_to_messages(history, approval_storage=resolved_storage) input_task = asyncio.ensure_future(_load_input()) history_task = asyncio.ensure_future(_load_history()) @@ -754,6 +798,7 @@ async def _handle_inner_agent( agent: SupportsAgentRun, configuration: _AgentConfiguration, hosted_request: ResponsesRequest, + deferred: _DeferredOutput, ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: """Handle a regular (non-workflow) agent. @@ -830,11 +875,6 @@ async def _handle_inner_agent( ) request_context = get_request_context() - approval_storage = ( - self._function_approval_storage_provider.get_store(config=self.config, platform_context=request_context) - if stored - else None - ) previous_response_id = request.get("previous_response_id") session_load_id = ( context.response_id @@ -848,26 +888,25 @@ async def _handle_inner_agent( ) session_storage = ( self._session_storage_provider.get_store(config=self.config, platform_context=request_context) - if stored or (session_load_id is not None and self._history_source == "agent_managed") + if stored or session_load_id is not None else None ) - - # Load the caller's input items and prior conversation history concurrently with the - # session load below. These are independent storage round-trips with no data dependency - # between them, so overlapping them removes serial latency from the request critical path. + # Reads start before the session loads; wire-ID translation waits for the session's approval map. + approval_ready: asyncio.Future[SessionApprovalState] = asyncio.get_running_loop().create_future() request_messages_task = asyncio.ensure_future( self._load_request_messages( context, - approval_storage=approval_storage, + approval_storage=None, configuration=configuration, + approval_ready=approval_ready, ) ) - - session = ( + stored_session = ( await session_storage.get(session_load_id) if session_storage is not None and session_load_id is not None else None ) + session = stored_session if stored or self._history_source == "agent_managed" else None if session is None: if context.is_recovery and provider_background: raise _PublicResponseFailure( @@ -880,6 +919,9 @@ async def _handle_inner_agent( ) ) session = agent.create_session() + # Core's session approval tracking binds and consumes replies; this map only translates wire IDs. + approval_storage = SessionApprovalState(session) + approval_ready.set_result(approval_storage) provider_state = session.state.get(_HOSTED_PROVIDER_STATE_KEY) if context.conversation_id is not None: if context.is_recovery and provider_background: @@ -940,8 +982,7 @@ async def _handle_inner_agent( # task so it is not orphaned, and log only ordinary failures. if request_messages_task is not None: request_messages_task.cancel() - with suppress(BaseException): - await request_messages_task + await asyncio.gather(request_messages_task, return_exceptions=True) if isinstance(ex, Exception): _log_response_failure("state storage preparation", context.response_id, ex) raise @@ -963,8 +1004,10 @@ async def _handle_inner_agent( "messages": messages, "session": session, } - chat_options = cast(ChatOptions[Any], dict(hosted_request.options)) + chat_options = dict(hosted_request.options) are_options_set = bool(chat_options) + if self._background_source == "inference_service": + chat_options.setdefault("background", provider_background) if configuration.agent_server_history: if configuration.client_stores_by_default: # The response provider already owns the transcript used for this run. Keep a @@ -987,13 +1030,8 @@ async def _handle_inner_agent( ): chat_options["store"] = False - if isinstance(agent, RawAgent): + if isinstance(agent, RawAgent) or are_options_set: run_kwargs["options"] = chat_options - elif are_options_set: - if self._unsupported_options == "error": - raise _PublicResponseFailure(TypeError("The hosted agent does not accept caller runtime options.")) - if self._unsupported_options == "warn": - logger.warning("Agent doesn't support runtime options. They will be ignored.") if previous_response_id is not None and context.conversation_id is None and not context.is_recovery: if ( @@ -1030,6 +1068,38 @@ async def _handle_inner_agent( finally: session.state.pop(_HOSTED_CONVERSATION_CLAIM_KEY, None) + consent_scope = _activate_toolbox_consent_scope(allowed_origins=self._allowed_oauth_consent_origins) + + async def track_update( + update: AgentResponseUpdate, *, complete_message: bool = False + ) -> AsyncGenerator[ResponseStreamEvent]: + if consent_scope.consent_rejected: + raise _PublicResponseFailure(ValueError("OAuth consent link rejected.")) + has_continuation = any( + content.type in ("function_approval_request", "oauth_consent_request") or content.user_input_request + for content in update.contents + ) + if not stored and (has_continuation or consent_scope.consent_required): + raise _PublicResponseFailure( + ValueError("OAuth consent or approval continuation requires store=true.") + ) + if self._history_source == "inference_service" and ( + consent_scope.consent_required + or any(content.type == "oauth_consent_request" for content in update.contents) + ): + raise _PublicResponseFailure( + ValueError( + "Invocation-time OAuth consent is unsupported with history_source='inference_service'." + ) + ) + if deferred.snapshot is None and has_continuation: + deferred.snapshot = deepcopy(response_event_stream.response) + async for event in tracker.handle_update( + update, approval_storage=approval_storage, complete_message=complete_message + ): + if not deferred.defer(event): + yield event + inner_stream: ResponseStream[AgentResponseUpdate, AgentResponse[Any]] | None = None updates: _SignalledIterator[AgentResponseUpdate] | None = None if provider_background: @@ -1045,7 +1115,7 @@ async def _handle_inner_agent( messages=messages, session=session, session_storage=session_storage, - options=chat_options, + options=cast(ChatOptions[Any], chat_options), context=context, cancellation_signal=cancellation_signal, emitted_output_count=output_count, @@ -1055,32 +1125,31 @@ async def _handle_inner_agent( for update in _agent_response_updates(response, context.response_id): if context.shutdown.is_set() or cancellation_signal.is_set(): break - async for event in tracker.handle_update( - update, approval_storage=approval_storage, complete_message=True - ): + async for event in track_update(update, complete_message=True): yield event if context.shutdown.is_set() or cancellation_signal.is_set(): continue for event in tracker.close(): - yield event + if not deferred.defer(event): + yield event private_metadata = _hosted_internal_metadata(response_event_stream) private_metadata[_HOSTED_PROVIDER_OUTPUT_COUNT_KEY] = output_count private_metadata[_HOSTED_PROVIDER_USAGE_KEY] = tracker.usage_details if self._resilient_background: - yield response_event_stream.checkpoint() + checkpoint = response_event_stream.checkpoint() + if not deferred.defer(checkpoint): + yield checkpoint else: inner_stream = agent.run(stream=True, **run_kwargs) # type: ignore[reportUnknownMemberType] updates = _SignalledIterator(inner_stream, context.shutdown, cancellation_signal) async with aclosing(updates): async for update in updates: - if not stored and any( - content.type in ("function_approval_request", "oauth_consent_request") - or content.user_input_request - for content in update.contents - ): - raise ValueError("Approval and user-input continuation requires store=true.") - async for event in tracker.handle_update(update, approval_storage=approval_storage): + async for event in track_update(update): yield event + if consent_scope.consent_rejected: + raise _PublicResponseFailure(ValueError("OAuth consent link rejected.")) + if consent_scope.consent_required and not stored: + raise _PublicResponseFailure(ValueError("OAuth consent continuation requires store=true.")) if inner_stream is not None and isinstance(updates, _SignalledIterator) and not updates.signalled: final = await inner_stream.get_final_response() if final.continuation_token is not None and final.finish_reason is None: @@ -1141,9 +1210,6 @@ async def _handle_inner_agent( ) ) ) - superseded_by_steering = bool(self._host_options and self._host_options.steerable_conversations) and ( - cancellation_signal.is_set() and not context.client_cancelled and not context.shutdown.is_set() - ) if stored and session_storage is not None and not stored_output_violation and final_provider_state: if context.conversation_id is not None: session.state[_HOSTED_SOURCE_CONVERSATION_KEY] = context.conversation_id @@ -1151,7 +1217,6 @@ async def _handle_inner_agent( if ( context.conversation_id is not None and not conversation_already_committed - and not superseded_by_steering and not request_interrupted and request_failure is None and ( @@ -1182,6 +1247,9 @@ async def _handle_inner_agent( raise request_failure elif save_failure is not None: raise save_failure + if not request_interrupted and not context.shutdown.is_set() and not cancellation_signal.is_set(): + for event in deferred.events: + yield event async def _provider_background_responses( self, @@ -1210,7 +1278,7 @@ async def proceed(*, has_token: bool) -> bool: if has_token and self._resilient_background: await context.exit_for_recovery() if has_token: - raise RuntimeError("Provider background recovery requires resilient_background=True.") + raise RuntimeError("Provider background recovery requires enable_background_recovery=True.") raise _PublicResponseFailure( RuntimeError( "Provider background submission stopped before its token was saved; cannot safely retry it." @@ -1224,7 +1292,11 @@ async def run_provider( options: ChatOptions[Any], *, input_messages: list[Message] | None, phase: Literal["submit", "poll"] ) -> AgentResponse[Any]: try: - return await agent.run(input_messages, session=session, options=options) + return await agent.run( + input_messages, + session=session, + options=options, + ) except Exception as exc: logger.warning("Inner provider background %s failed (%s).", phase, type(exc).__name__) raise RuntimeError(f"Inner provider background {phase} failed; inspect the host logs.") diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py index 1298d66ad85..29b29c4e166 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py @@ -21,8 +21,9 @@ Sequence, ) from contextlib import suppress -from dataclasses import asdict, dataclass, is_dataclass -from typing import Any, Generic, Literal, TypeGuard, TypeVar, cast +from contextvars import ContextVar +from dataclasses import asdict, dataclass, field, is_dataclass +from typing import Any, Generic, Literal, Protocol, TypeGuard, TypeVar, cast from urllib.parse import urlparse from agent_framework import ( @@ -45,10 +46,7 @@ from agent_framework._mcp import _MCP_TOOL_RESULT_HOST_PAYLOAD_KEY # pyright: ignore[reportPrivateUsage] from agent_framework._types import _get_annotation_text_span_regions # pyright: ignore[reportPrivateUsage] from agent_framework.observability import OBSERVABILITY_SETTINGS -from azure.ai.agentserver.responses import ( - ResponseContext, - ResponsesServerOptions, -) +from azure.ai.agentserver.responses import ResponseContext from azure.ai.agentserver.responses._id_generator import IdGenerator from azure.ai.agentserver.responses.aio import ResponseEventStream from azure.ai.agentserver.responses.models import ( @@ -94,12 +92,23 @@ ) from mcp import MCPError -from ._state_store import ( - FunctionApprovalStore, -) +from ._approvals import SessionApprovalState, _InvalidApproval # pyright: ignore[reportPrivateUsage] logger = logging.getLogger(__name__) + +class _ApprovalRecorder(Protocol): + """Records the wire ID of each approval item before the output tracker yields it. + + The mapping only translates IDs. The host must buffer the yielded events until the state that holds the + mapping is saved. + """ + + async def save_approval_request(self, approval_request_id: str, request: Content) -> None: + """Record the original ``function_approval_request`` content under its generated public ID.""" + ... + + _MODEL_OUTPUT_KIND_KEY = "model_output_kind" _MODEL_OUTPUT_REFUSAL = "refusal" _HOSTED_PROVIDER_STATE_KEY = "_foundry_provider_background" @@ -786,19 +795,104 @@ def consent_url_from_error(exc: BaseException) -> list[ConsentError] | None: return None -# endregion Foundry Toolbox Auth integration +_REMOTE_ERROR_PREFIX = "Request failed (remote): " +OAUTH_CONSENT_REQUIRED_RESULT = ( + "Error: OAuth consent is required. This tool invocation did not complete. The user must authorize access " + "before this tool can be invoked in a later request." +) +OAUTH_CONSENT_NOT_EXECUTED_RESULT = ( + "Error: This tool invocation was not executed because another tool requires OAuth consent. The user must " + "authorize access before protected tools can be invoked in a later request." +) -def _validate_response_server_options( # pyright: ignore[reportUnusedFunction] - options: ResponsesServerOptions | None, -) -> None: - if options and options.steerable_conversations: - # TODO(foundry-hosting): Remove after Azure/azure-sdk-for-python#49233 ships and queue overflow is verified. - raise RuntimeError( - "steerable_conversations=True is temporarily unavailable: the current AgentServer SDK " - "retains futures for rejected steering turns. Wait for the official fix in " - "Azure/azure-sdk-for-python#49233, then update the dependency and verify queue overflow." - ) + +def tool_call_consent_url_from_error(exc: BaseException) -> str | None: + """Return the consent URL from a Foundry Toolbox ``tools/call`` consent error, if ``exc`` wraps one. + + The gateway reports invocation-time consent as MCP error ``-32006``. The URL is read from a string + ``error.data["detail"]``; when ``detail`` is absent, the error message is used after removing the + ``"Request failed (remote): "`` prefix. A ``detail`` value that is present but not a string yields an empty + URL so the caller fails closed. The ``tools/list`` JSON ``errors`` shape is also accepted. + + Args: + exc: The exception to inspect. ``exc`` itself or one of its ``args`` must be the ``MCPError``. + + Returns: + The raw consent URL (not yet validated), an empty string when the payload is malformed, or ``None`` when + ``exc`` is not a consent error. + """ + inner_exception = ( + exc if isinstance(exc, MCPError) else next((arg for arg in exc.args if isinstance(arg, MCPError)), None) + ) + if inner_exception is None or inner_exception.code != CONSENT_ERROR_CODE: + return None + data = inner_exception.error.data + if isinstance(data, Mapping) and "detail" in data: + detail = cast(Mapping[str, Any], data)["detail"] + return detail.strip() if isinstance(detail, str) else "" + message = inner_exception.error.message or "" + if "{" in message: + listed = consent_url_from_error(Exception(inner_exception)) + return listed[0].consent_url if listed else "" + message = message.removeprefix(_REMOTE_ERROR_PREFIX) + return message.strip() + + +@dataclass +class _ToolboxConsentScope: + """Request-wide invocation-time OAuth consent state shared by all tool-call tasks of one hosted request. + + The scope is a mutable object so state recorded in a copied task context remains visible to the host and to + later tool calls. Hosts must create a fresh scope for every request. + """ + + allowed_origins: frozenset[str] | None = None + consents: list[ConsentError] = field(default_factory=lambda: []) + consent_rejected: bool = False + + @property + def consent_required(self) -> bool: + """Whether a protected tool in this request requires OAuth consent.""" + return bool(self.consents) + + @property + def blocks_toolbox_calls(self) -> bool: + """Whether later Foundry Toolbox calls in this request must not contact the gateway.""" + return self.consent_required or self.consent_rejected + + +_toolbox_consent_scope: ContextVar[_ToolboxConsentScope | None] = ContextVar( + "_foundry_toolbox_consent_scope", default=None +) + + +def _activate_toolbox_consent_scope( # pyright: ignore[reportUnusedFunction] + *, allowed_origins: frozenset[str] | None = None +) -> _ToolboxConsentScope: + """Activate a fresh invocation-time OAuth consent scope in the current context. + + Call this in the request task before the agent run starts and before the first stream item is pulled, so all + tool-call tasks inherit the scope. The scope is intentionally not reset: async generator finalization may run + in a different context, and a fresh scope replaces it for the next request. + + Args: + allowed_origins: Normalized consent origin allowlist from ``_normalize_allowed_oauth_consent_origins``. + + Returns: + The active scope. + """ + scope = _ToolboxConsentScope(allowed_origins=allowed_origins) + _toolbox_consent_scope.set(scope) + return scope + + +def _current_toolbox_consent_scope() -> _ToolboxConsentScope | None: # pyright: ignore[reportUnusedFunction] + """Return the active invocation-time OAuth consent scope, if a host activated one.""" + return _toolbox_consent_scope.get() + + +# endregion Foundry Toolbox Auth integration # region Active Builder State @@ -983,7 +1077,7 @@ async def handle_update( self, update: AgentResponseUpdate, *, - approval_storage: FunctionApprovalStore | None = None, + approval_storage: _ApprovalRecorder | None = None, complete_message: bool = False, ) -> AsyncGenerator[ResponseStreamEvent]: """Process one agent update: note its finish reason, then handle each of its contents. @@ -1008,7 +1102,7 @@ async def handle( content: Content, message_id: str | None = None, *, - approval_storage: FunctionApprovalStore | None = None, + approval_storage: _ApprovalRecorder | None = None, start_new_content_part: bool = False, ) -> AsyncGenerator[ResponseStreamEvent]: """Process a content item, yielding its events. @@ -1019,8 +1113,9 @@ async def handle( change in ``message_id`` across otherwise same-typed text content marks a new logical message and forces the previous output item closed, rather than being merged into it. - approval_storage: Used for content types that fall back to one-shot emission - (anything not recognized as a streaming delta type) to save/load approval requests. + approval_storage: Recorder that receives each ``function_approval_request`` under its generated + public ID before the approval item event is yielded. The host must buffer yielded approval + events until the state that holds the mapping is saved. start_new_content_part: Whether text and refusal content start a new independent content part instead of continuing the current streaming occurrence. """ @@ -1281,15 +1376,14 @@ async def handle( if isinstance(item, Mapping) else getattr(item, "id", None) ) - if isinstance(approval_request_id, str): - await approval_storage.save_approval_request(approval_request_id, content) - request_saved = True + if not isinstance(approval_request_id, str) or not approval_request_id: + raise RuntimeError( + "The approval request ID could not be extracted from the stream event, " + "so the approval request could not be recorded." + ) + await approval_storage.save_approval_request(approval_request_id, content) + request_saved = True yield event - if approval_storage is not None and not request_saved: - logger.warning( - "Approval request was not saved to approval storage because the approval request ID " - "could not be extracted from the stream event." - ) elif content.type == "oauth_consent_request": for event in self._close(): @@ -1586,21 +1680,24 @@ def _persist_pending_container_file_citations(self) -> None: async def _items_to_messages( # pyright: ignore[reportUnusedFunction] - input_items: Sequence[Item], *, approval_storage: FunctionApprovalStore | None = None + input_items: Sequence[Item], *, approval_storage: SessionApprovalState | None = None ) -> list[Message]: """Converts a sequence of input items to a list of Messages, one per item. Args: input_items: The input items to convert. - approval_storage: An optional ApprovalStorage instance used to look up - approval requests when converting MCP approval response items. + approval_storage: Optional session-backed approval state used to read + approval requests without consuming their authority. Returns: A list of Messages, one per supported input item. """ messages: list[Message] = [] for item in input_items: - messages.append(await _item_to_message(item, approval_storage=approval_storage)) + try: + messages.append(await _item_to_message(item, approval_storage=approval_storage)) + except _InvalidApproval as exc: + _log_dropped_approval(exc) return messages @@ -1692,18 +1789,29 @@ def _shell_command_output_to_content(output: Mapping[str, Any]) -> Content: ) +async def _load_approval_request(approval_storage: SessionApprovalState | None, wire_id: str) -> Content: + if approval_storage is None: + raise ValueError("ApprovalStorage is required to load approval request.") + return await approval_storage.load_approval_request(wire_id) + + +def _log_dropped_approval(exc: _InvalidApproval) -> None: + # Core authorizes approvals against the session's pending requests; an unknown wire ID is dropped. + logger.warning("Ignoring an MCP approval item: %s", exc) + + async def _item_to_message( item: Item, *, - approval_storage: FunctionApprovalStore | None = None, + approval_storage: SessionApprovalState | None = None, _item_type_name: Literal["Item", "OutputItem"] = "Item", ) -> Message: """Converts an Item to a Message. Args: item: The Item to convert. - approval_storage: An optional ApprovalStorage instance used to look up - approval requests when converting MCP approval response items. + approval_storage: Optional session-backed approval state used to read + approval requests without consuming their authority. _item_type_name: The item type name to include in unsupported-type errors. Returns: @@ -1711,6 +1819,7 @@ async def _item_to_message( Raises: ValueError: If the Item type is not supported. + _InvalidApproval: If an MCP approval item's wire ID is not in the session. """ if item["type"] == "message": if isinstance(item["content"], str): @@ -1787,32 +1896,12 @@ async def _item_to_message( ) if item["type"] == "mcp_approval_request": - if approval_storage is not None: - try: - function_approval_request_content = await approval_storage.load_approval_request(item["id"]) - except (KeyError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - else: - raise ValueError("ApprovalStorage is required to load approval request.") - return Message( - role="assistant", - contents=[function_approval_request_content], - ) + request_content = await _load_approval_request(approval_storage, item["id"]) + return Message(role="assistant", contents=[request_content]) if item["type"] == "mcp_approval_response": - if approval_storage is not None: - try: - function_approval_request_content = await approval_storage.load_approval_request( - item["approval_request_id"] - ) - except (KeyError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - else: - raise ValueError("ApprovalStorage is required to load approval request.") - return Message( - role="user", - contents=[function_approval_request_content.to_function_approval_response(item["approve"])], - ) + request_content = await _load_approval_request(approval_storage, item["approval_request_id"]) + return Message(role="user", contents=[request_content.to_function_approval_response(item["approve"])]) if item["type"] == "code_interpreter_call": return Message( @@ -1999,39 +2088,41 @@ async def _item_to_message( async def _output_items_to_messages( # pyright: ignore[reportUnusedFunction] history: Sequence[OutputItem], *, - approval_storage: FunctionApprovalStore | None = None, + approval_storage: SessionApprovalState | None = None, ) -> list[Message]: """Converts a sequence of OutputItem objects to a list of Message objects. Args: history (Sequence[OutputItem]): The sequence of OutputItem objects to convert. - approval_storage (ApprovalStorage | None, optional): The approval storage to use for - resolving MCP approval requests. Defaults to None. + approval_storage: Optional session-backed approval state used to read + approval requests without consuming their authority. Returns: list[Message]: The list of Message objects. """ messages: list[Message] = [] for item in history: - messages.append(await _output_item_to_message(item, approval_storage=approval_storage)) + try: + messages.append(await _output_item_to_message(item, approval_storage=approval_storage)) + except _InvalidApproval as exc: + _log_dropped_approval(exc) return messages -async def _output_item_to_message( - item: OutputItem, *, approval_storage: FunctionApprovalStore | None = None -) -> Message: +async def _output_item_to_message(item: OutputItem, *, approval_storage: SessionApprovalState | None = None) -> Message: """Converts an OutputItem to a Message. Args: item (OutputItem): The OutputItem to convert. - approval_storage (ApprovalStorage | None, optional): The approval storage to use for - resolving MCP approval requests. Defaults to None. + approval_storage: Optional session-backed approval state used to read + approval requests without consuming their authority. Returns: Message: The converted Message. Raises: ValueError: If the OutputItem type is not supported. + _InvalidApproval: If an MCP approval item's wire ID is not in the session. """ if item["type"] == "oauth_consent_request": item_data = cast("Mapping[str, Any]", item) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py index 31f148331df..58eb4746bad 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py @@ -32,11 +32,12 @@ from agent_framework._telemetry import mark_feature_used from anyio import CancelScope from azure.ai.agentserver.core import get_request_context -from azure.ai.agentserver.responses import ResponseContext, ResponseProviderProtocol, ResponsesServerOptions +from azure.ai.agentserver.responses import ResponseContext, ResponseProviderProtocol from azure.ai.agentserver.responses._id_generator import IdGenerator from azure.ai.agentserver.responses.aio import ResponseEventStream from azure.ai.agentserver.responses.hosting import ResponsesAgentServerHost from azure.ai.agentserver.responses.models import CreateResponse, ResponseObject, ResponseStreamEvent +from azure.ai.agentserver.responses.models._helpers import get_input_expanded from azure.ai.agentserver.responses.streaming._checkpoint import ResponseCheckpointEvent from pydantic import BaseModel from starlette.requests import Request @@ -54,19 +55,16 @@ _OutputItemTracker, # pyright: ignore[reportPrivateUsage] _request_without_reserved_metadata, # pyright: ignore[reportPrivateUsage] _SignalledIterator, # pyright: ignore[reportPrivateUsage] - _validate_response_server_options, # pyright: ignore[reportPrivateUsage] ) from ._scope import FoundryRequestScope from ._shared import ( - OptionsHook, ResponsesRequest, WorkflowResolver, WorkflowSource, WorkflowTurn, - prepare_response_options, + build_workflow_response_server_options, prepare_workflow_kwargs, response_run_options, - validate_request_options, validate_workflow_source, workflow_agents, ) @@ -162,7 +160,9 @@ def _snapshot(stream: ResponseEventStream) -> dict[str, Any]: async def _load_replies(request: ResponsesRequest, run: HostedWorkflowRun) -> dict[str, Any]: items = await request.get_input_items() if not run.pending_requests or not items: - raise _WorkflowRequestError("Workflow replies require an exact pending stored checkpoint.") + raise _WorkflowRequestError("Workflow replies require a pending stored checkpoint.") + if len(get_input_expanded(cast(CreateResponse, request.request))) != len(items): + raise _WorkflowRequestError("Pending workflow replies cannot include unresolved input references.") replies: dict[str, Any] = {} for input_item in items: item = cast(Mapping[str, Any], input_item) @@ -172,12 +172,11 @@ async def _load_replies(request: ResponsesRequest, run: HostedWorkflowRun) -> di raise _WorkflowRequestError("An approval reply requires its exact ID and a boolean decision.") request_id = run.approvals.get(wire_id) pending = run.pending_requests.get(request_id) if request_id is not None else None - if ( - pending is None - or not isinstance(pending.data, Content) - or pending.data.type != "function_approval_request" - ): - raise _WorkflowRequestError("The approval is unavailable in this exact pending checkpoint.") + if request_id is None or pending is None: + logger.warning("Ignoring an approval reply that is not pending in this workflow checkpoint.") + continue + if not isinstance(pending.data, Content) or pending.data.type != "function_approval_request": + raise _WorkflowRequestError("An approval reply cannot answer a non-approval workflow request.") response = pending.data.to_function_approval_response(decision) elif item.get("type") in ("function_call_output", "computer_call_output"): call_id = item.get("call_id") @@ -185,7 +184,8 @@ async def _load_replies(request: ResponsesRequest, run: HostedWorkflowRun) -> di raise _WorkflowRequestError("A workflow reply requires its pending call ID.") request_id = run.approvals.get(call_id) if request_id is None or request_id not in run.pending_requests: - raise _WorkflowRequestError("A reply must match its wire ID in the exact pending workflow checkpoint.") + logger.warning("Ignoring a workflow reply that is not pending in this workflow checkpoint.") + continue pending = run.pending_requests[request_id] if isinstance(pending.data, Content) and pending.data.type == "function_approval_request": raise _WorkflowRequestError("An approval cannot be replaced by a function result.") @@ -216,11 +216,9 @@ async def _load_replies(request: ResponsesRequest, run: HostedWorkflowRun) -> di raise _WorkflowRequestError("A typed workflow reply must contain valid JSON.") from exc else: raise _WorkflowRequestError("Pending workflow replies cannot be mixed with new input.") - if request_id is None or request_id in replies: + if request_id in replies: raise _WorkflowRequestError("Duplicate workflow replies are not allowed.") replies[request_id] = response - if set(replies) != set(run.pending_requests): - raise _WorkflowRequestError("Answer exactly the complete pending workflow request batch.") validated = run.validate_turn(WorkflowTurn(responses=replies)) return dict(validated.responses or {}) @@ -242,13 +240,6 @@ async def save_approval_request(self, approval_request_id: str, request: Content raise _WorkflowRequestError("Approval output must match one unique request in the exact pause checkpoint.") self.aliases[approval_request_id] = matches[0] - async def load_approval_request(self, approval_request_id: str) -> Content: - request_id = self.aliases.get(approval_request_id) - event = self.pending.get(request_id) if request_id is not None else None - if event is None or not isinstance(event.data, Content): - raise _WorkflowRequestError("The approval is unavailable in this pending checkpoint.") - return event.data - async def _project( event: WorkflowEvent[Any], @@ -337,11 +328,12 @@ def __init__( *, parse_input: Callable[[ResponsesRequest], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], prefix: str = "", - options: ResponsesServerOptions | None = None, response_store: ResponseProviderProtocol | None = None, checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - prepare_options: OptionsHook | None = None, allowed_oauth_consent_origins: Sequence[str] | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + enable_background_recovery: bool = False, **kwargs: Any, ) -> None: """Create a native workflow host with explicit parsing and checkpoint ownership. @@ -356,51 +348,39 @@ def __init__( parse_input: Sync or async parser of this turn's `ResponsesRequest`. Return a typed `WorkflowTurn(input=...)` or a complete batch of validated pending `responses`. prefix: URL prefix for Responses routes. - options: AgentServer options, including outer background recovery. Steering remains disabled. - Outer background execution does not enable provider background on workflow agents. response_store: Backend for caller-facing Responses persistence, not the wire-level `store` flag. checkpoint_store_provider: Scoped checkpoint storage. Defaults to `CheckpointStoreProvider`. Native workflow history and pending replies are checkpoint-owned, not agent-session-owned. - prepare_options: Sync or async hook receiving the hosted request and a copy of caller model - options before parsing/execution. Return the replacement mapping; hosting validates it. - This does not enable inner provider background or service-owned history. allowed_oauth_consent_origins: Optional exact HTTPS origins for consent links. Omission retains safe-HTTPS validation; an empty sequence rejects all origins. - **kwargs: Additional AgentServer host settings forwarded to the SDK. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval. If omitted, the SDK + uses a positive `SSE_KEEPALIVE_INTERVAL` environment value, or disables heartbeats. + shutdown_grace_period_seconds: Positive shutdown wait, also used by the SDK when DELETE waits + for response persistence. Defaults to 10 seconds. + enable_background_recovery: Allow checkpoint-backed outer workflow recovery. Defaults to `False`. + Outer background execution does not enable provider background on workflow agents. + **kwargs: Additional AgentServer SDK constructor arguments. Raises: - TypeError: If the workflow is not a request-aware factory, the parser is not callable, or an - agent-only setting is supplied. - RuntimeError: If steering is requested while unavailable. + TypeError: If the workflow is not a request-aware factory or the parser is not callable. + ValueError: If a supported setting has an invalid value. """ - if "parse_response" in kwargs: - raise TypeError("parse_response was renamed to parse_input.") - forbidden = { - "agent", - "agent_session_store_provider", - "function_approval_store_provider", - "history_source", - "background_source", - "unsupported_options", - "store", - }.intersection(kwargs) - if forbidden: - raise TypeError(f"Unsupported native workflow host settings: {', '.join(sorted(forbidden))}.") validate_workflow_source(workflow) if not callable(parse_input): raise TypeError("parse_input must be a callable.") - if prepare_options is not None and not callable(prepare_options): - raise TypeError("prepare_options must be a callable.") - _validate_response_server_options(options) allowed_origins = _normalize_allowed_oauth_consent_origins(allowed_oauth_consent_origins) + options = build_workflow_response_server_options( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + enable_background_recovery=enable_background_recovery, + ) super().__init__(prefix=prefix, options=options, store=response_store, **kwargs) self._resolver = WorkflowResolver(workflow) self._parse_input = parse_input self._checkpoint_store_provider = ( CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider ) - self._prepare_options = prepare_options - self._resilient_background = bool(options and options.resilient_background) + self._resilient_background = enable_background_recovery self._allowed_origins = allowed_origins self._bind_streaming_route( prefix=prefix, @@ -413,7 +393,10 @@ def __init__( def _bind_streaming_route(self, *, prefix: str, keep_alive: bool) -> None: """Adapt only the native POST route's public StreamingResponse cleanup lifetime.""" - path = f"/{prefix.strip('/')}/responses" if prefix.strip("/") else "/responses" + normalized_prefix = prefix.strip() + if normalized_prefix and not normalized_prefix.startswith("/"): + normalized_prefix = f"/{normalized_prefix}" + path = f"{normalized_prefix.rstrip('/')}/responses" for index, route in enumerate(self.router.routes): if not isinstance(route, Route) or route.path != path or "POST" not in (route.methods or ()): continue @@ -498,8 +481,6 @@ async def abort() -> bool: self.config, platform_context, local_session_id=context.response_id ) hosted = ResponsesRequest(request, context, scope, response_run_options(request)) - await prepare_response_options(hosted, self._prepare_options) - validate_request_options(hosted.options) workflow = await self._resolver.resolve(hosted) resources = AsyncExitStack() await resources.__aenter__() @@ -556,7 +537,6 @@ async def abort() -> bool: workflow, turn, scope, - options=hosted.options, stored=run.stored, fresh_factory=True, ) @@ -628,6 +608,18 @@ async def checkpoint_stamp() -> str | None: # ruff: ignore[unused-async] await abort() return phase = "final output persistence" + if run.stored and not run.has_completed_output: + # Core does not re-emit requests left unanswered by a partial reply; project them under new wire IDs. + checkpoint = await run.get_checkpoint(run.checkpoint_id) + projected = set(aliases.values()) | { + event.request_id for event in buffered if event.type == "request_info" + } + if checkpoint is not None: + buffered.extend( + event + for request_id, event in checkpoint.pending_request_info_events.items() + if request_id not in projected + ) if buffered: for event in await flush(run.checkpoint_id): yield wire(event) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py index 504bc3b0a8b..604615c02aa 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py @@ -5,6 +5,7 @@ from __future__ import annotations import inspect +import warnings import weakref from collections import OrderedDict from collections.abc import Awaitable, Callable, Iterator, Mapping, Sequence @@ -14,6 +15,7 @@ from typing import Any, Generic, Literal, Protocol, TypeAlias, TypeGuard, cast from agent_framework import ( + SKIP_PARSING, Agent, AgentContext, AgentExecutor, @@ -21,6 +23,7 @@ AgentResponse, AgentRunInputs, Content, + FunctionTool, HistoryProvider, InMemoryHistoryProvider, Message, @@ -33,13 +36,109 @@ WorkflowInvocationKwargs, WorkflowRunState, ) -from azure.ai.agentserver.responses import ResponseContext +from azure.ai.agentserver.responses import ResponseContext, ResponsesServerOptions from azure.ai.agentserver.responses.models import CreateResponse, Item from typing_extensions import TypeVar from ._scope import FoundryRequestScope -UnsupportedOptions: TypeAlias = Literal["ignore", "warn", "error"] + +def validate_sse_keep_alive_interval(value: int | None) -> None: + """Validate an explicit heartbeat interval without replacing SDK environment resolution.""" + if value is not None: + if isinstance(value, bool) or not isinstance(value, int): + raise TypeError("sse_keep_alive_interval_seconds must be an integer or None.") + if value <= 0: + raise ValueError("sse_keep_alive_interval_seconds must be positive when supplied.") + + +def _validate_shared_response_settings( + *, + sse_keep_alive_interval_seconds: int | None, + shutdown_grace_period_seconds: int, + enable_background_recovery: bool, +) -> None: + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + if isinstance(shutdown_grace_period_seconds, bool) or not isinstance(shutdown_grace_period_seconds, int): + raise TypeError("shutdown_grace_period_seconds must be an integer.") + if shutdown_grace_period_seconds <= 0: + raise ValueError("shutdown_grace_period_seconds must be positive.") + if not isinstance(enable_background_recovery, bool): + raise TypeError("enable_background_recovery must be a boolean.") + + +def build_agent_response_server_options( + *, + history_source: Literal["host_managed", "agent_managed", "inference_service"], + background_source: Literal["host_managed", "inference_service"], + max_history_items: int | None, + sse_keep_alive_interval_seconds: int | None, + shutdown_grace_period_seconds: int, + enable_background_recovery: bool, +) -> ResponsesServerOptions: + """Validate Responses agent host settings and build SDK options without host initialization. + + Omit an unset history limit so the installed SDK supplies its own default. + Leave heartbeat environment resolution to the SDK: an explicit positive + interval wins over a positive ``SSE_KEEPALIVE_INTERVAL``, otherwise disabled. + """ + _validate_shared_response_settings( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + enable_background_recovery=enable_background_recovery, + ) + if history_source not in ("host_managed", "agent_managed", "inference_service"): + raise ValueError("history_source must be 'host_managed', 'agent_managed', or 'inference_service'.") + if background_source not in ("host_managed", "inference_service"): + raise ValueError("background_source must be 'host_managed' or 'inference_service'.") + if background_source == "inference_service" and history_source != "inference_service": + raise ValueError("background_source='inference_service' requires history_source='inference_service'.") + if enable_background_recovery and background_source != "inference_service": + raise ValueError( + "enable_background_recovery=True requires history_source='inference_service' and " + "background_source='inference_service'." + ) + kwargs: dict[str, Any] = {} + if max_history_items is not None: + if isinstance(max_history_items, bool) or not isinstance(max_history_items, int): + raise TypeError("max_history_items must be an integer or None.") + if max_history_items <= 0: + raise ValueError("max_history_items must be positive when supplied.") + if history_source != "host_managed": + warnings.warn( + "max_history_items is ignored unless history_source='host_managed'.", + UserWarning, + stacklevel=3, + ) + else: + kwargs["default_fetch_history_count"] = max_history_items + return ResponsesServerOptions( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=enable_background_recovery, + steerable_conversations=False, + **kwargs, + ) + + +def build_workflow_response_server_options( + *, + sse_keep_alive_interval_seconds: int | None, + shutdown_grace_period_seconds: int, + enable_background_recovery: bool, +) -> ResponsesServerOptions: + """Validate Responses workflow host settings and build SDK options without host initialization.""" + _validate_shared_response_settings( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + enable_background_recovery=enable_background_recovery, + ) + return ResponsesServerOptions( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=enable_background_recovery, + steerable_conversations=False, + ) InputT = TypeVar("InputT", default=Any) @@ -141,6 +240,17 @@ def __post_init__(self) -> None: _OPTION_NAMES = {"max_output_tokens": "max_tokens", "parallel_tool_calls": "allow_multiple_tool_calls"} +_RUN_CONTROLLED_FIELDS = frozenset({ + "additional_function_arguments", + "client_kwargs", + "compaction_strategy", + "function_invocation_kwargs", + "messages", + "middleware", + "session", + "tokenizer", +}) + _NATIVE_FIELDS = frozenset(CreateResponse.__annotations__) @@ -160,6 +270,8 @@ def response_run_options(request: CreateResponse) -> dict[str, Any]: raise TypeError("extra_body must be a mapping of model options.") if "extra_body" in value: raise ValueError("Nested extra_body is not supported; flatten provider options instead.") + if reserved := _HOST_CONTROLLED_FIELDS.intersection(cast(Mapping[str, Any], value)): + raise ValueError(f"extra_body cannot set host-controlled fields: {', '.join(sorted(reserved))}.") nested_extra.update(cast(Mapping[str, Any], value)) elif name in _HOST_CONTROLLED_FIELDS: continue @@ -170,7 +282,7 @@ def response_run_options(request: CreateResponse) -> dict[str, Any]: return { **native, **{name: value for name, value in extra.items() if name not in _HOST_CONTROLLED_FIELDS}, - **{name: value for name, value in nested_extra.items() if name not in _HOST_CONTROLLED_FIELDS}, + **nested_extra, } @@ -194,7 +306,7 @@ def __init__( @property def options(self) -> Mapping[str, Any]: - """Return this turn's effective caller options after the hook.""" + """Return this turn's caller options, replaced by the ``prepare_options`` result on agent hosts.""" return self._options def set_options(self, options: Mapping[str, Any]) -> None: @@ -231,21 +343,26 @@ def _set_workflow_responses(self, loader: Callable[[], Awaitable[dict[str, Any]] async def prepare_response_options(request: ResponsesRequest, hook: OptionsHook | None) -> None: - """Apply a synchronous or asynchronous developer hook to a copy of caller options.""" + """Apply a sync/async option selection hook without changing agent defaults. + + Caller options are omitted when no hook is configured. The host validates the + hook's replacement mapping before dispatch. + """ if hook is None: + request.set_options({}) return - result = hook(request, dict(request.options)) + result = hook(request, deepcopy(dict(request.options))) if inspect.isawaitable(result): result = await result - if not isinstance(result, Mapping): + if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): raise TypeError("prepare_options must return a mapping of MAF run options.") - request.set_options(result) + request.set_options(deepcopy(dict(result))) async def response_input_messages(request: ResponsesRequest) -> list[Message]: - """Convert only this Responses turn's input items to the legacy workflow message contract. + """Convert only this Responses turn's input items to `list[Message]` workflow input. - This explicit migration helper does not load outer history, restore checkpoints, + This explicit helper does not load outer history, restore checkpoints, or decode pending native workflow replies. Native workflows should prefer typed application inputs and use ``request.get_workflow_responses()`` for resumable pauses. """ @@ -254,13 +371,55 @@ async def response_input_messages(request: ResponsesRequest) -> list[Message]: return await _items_to_messages(await request.get_input_items(), approval_storage=None) -def validate_request_options(options: Mapping[str, Any]) -> None: - """Keep hosting identity, storage decisions, and private continuation out of model options.""" - reserved = _HOST_CONTROLLED_FIELDS.intersection(options) +def validate_request_options(options: Mapping[str, Any], *, allow_background: bool = False) -> None: + """Keep hosting identity, storage decisions, and private continuation out of model options. + + With ``allow_background``, a ``background`` value set by the application's + options hook is passed to the agent unchanged; the application owns that choice. + """ + if not isinstance(options, Mapping) or any(not isinstance(key, str) for key in options): + raise TypeError("Run options must be a mapping with string keys.") + reserved = set((_HOST_CONTROLLED_FIELDS | _RUN_CONTROLLED_FIELDS).intersection(options)) + if allow_background: + reserved.discard("background") if reserved: raise ValueError(f"prepare_options cannot set host-controlled fields: {', '.join(sorted(reserved))}.") +def validate_agent_background_options(agent: SupportsAgentRun, *, provider_background: bool) -> None: + """Reject hidden provider background in hosting-visible agent defaults without mutating them. + + A selected regular-agent provider-background request may use a top-level + boolean default. A transport ``extra_body`` override is never supported. + Clients hard-wired internally to provider background are unsupported. + This does not select execution mode or validate client/storage capabilities. + """ + defaults = getattr(agent, "default_options", {}) + if not isinstance(defaults, Mapping): + raise TypeError("Agent default_options must be a mapping.") + validate_provider_background_options(cast(Mapping[str, Any], defaults), provider_background=provider_background) + + +def validate_provider_background_options(options: Mapping[str, Any], *, provider_background: bool) -> None: + """Validate visible provider-background options without changing the execution mode. + + Permit a boolean top-level default only when compatible with the selected + mode. Reject transport-body overrides even when false. This pure check does + not inspect client internals or produce protocol responses. + """ + background = options.get("background") + if background is not None and not isinstance(background, bool): + raise TypeError("Provider background must be a boolean.") + if background is True and not provider_background: + raise ValueError( + "Provider background=True requires an outer background request with " + "history_source='inference_service' and background_source='inference_service'." + ) + extra_body = options.get("extra_body") + if isinstance(extra_body, Mapping) and "background" in extra_body: + raise ValueError("Provider extra_body cannot set host-controlled fields: background.") + + def validate_default_transport_options(defaults: Mapping[str, Any], *, allow_agent_store: bool) -> None: """Reject transport overrides that would bypass the host's inner storage and identity decisions.""" extra_body = defaults.get("extra_body") @@ -280,13 +439,6 @@ def validate_default_transport_options(defaults: Mapping[str, Any], *, allow_age ) -def validate_unsupported_options(mode: str) -> UnsupportedOptions: - """Reject misspelled unsupported-options policies at host construction.""" - if mode not in ("ignore", "warn", "error"): - raise ValueError("unsupported_options must be 'ignore', 'warn', or 'error'.") - return mode - - RequestT = TypeVar("RequestT") AgentRequestT = TypeVar("AgentRequestT", contravariant=True) @@ -296,7 +448,7 @@ class AgentFactory(Protocol[AgentRequestT]): def __call__(self, *, request: AgentRequestT) -> SupportsAgentRun | Awaitable[SupportsAgentRun]: """Create an agent, optionally using the current request.""" - ... + raise NotImplementedError AgentSource: TypeAlias = SupportsAgentRun | AgentFactory[RequestT] @@ -366,38 +518,41 @@ async def resolve_agent(source: AgentSource[RequestT], request: RequestT) -> Sup _MAX_STRONG_RESOURCE_IDENTITIES = 1024 -_CLIENT_CONTROLS = frozenset({ - "additional_function_arguments", - "agent", - "background", - "call_id", - "checkpoint_id", - "checkpoint_storage", - "client_kwargs", - "continuation_token", - "conversation", - "conversation_id", - "extra_body", - "function_invocation_kwargs", - "fresh_factory", - "input", - "instructions", - "messages", - "middleware", - "previous_response_id", - "response_id", - "responses", - "service_session_id", - "session", - "session_id", - "agent_session_id", - "store", - "stream", - "tokenizer", - "tools", - "user", - "user_id", -}) +_CLIENT_CONTROLS = ( + frozenset({ + "additional_function_arguments", + "agent", + "background", + "call_id", + "checkpoint_id", + "checkpoint_storage", + "client_kwargs", + "continuation_token", + "conversation", + "conversation_id", + "extra_body", + "function_invocation_kwargs", + "fresh_factory", + "input", + "instructions", + "messages", + "middleware", + "previous_response_id", + "response_id", + "responses", + "service_session_id", + "session", + "session_id", + "agent_session_id", + "store", + "stream", + "tokenizer", + "tools", + "user", + "user_id", + }) + | _RUN_CONTROLLED_FIELDS +) class _NativeAgentOptions(AgentMiddleware): @@ -416,6 +571,12 @@ async def process(self, context: AgentContext, call_next: Callable[[], Awaitable if context.session is not None and context.session.service_session_id is not None: raise RuntimeError("Native workflow checkpoints cannot resume a private downstream service session.") effective = {**(context.options or {}), **options} + if (_CLIENT_CONTROLS - {"store", "instructions", "tools", "extra_body"}).intersection(effective): + raise ValueError("Native workflow model options cannot override hosting or transport controls.") + if (_CLIENT_CONTROLS - {"additional_function_arguments", "middleware"}).intersection(context.client_kwargs): + raise ValueError("Native workflow client kwargs cannot override hosting or transport controls.") + validate_default_transport_options(effective, allow_agent_store=False) + validate_agent_background_options(context.agent, provider_background=False) client = getattr(context.agent, "client", None) if getattr(client, "STORES_BY_DEFAULT", None) is True: effective["store"] = False @@ -447,7 +608,6 @@ def prepare_workflow_kwargs( turn: WorkflowTurn[Any], scope: FoundryRequestScope, *, - options: Mapping[str, Any] | None = None, stored: bool = True, fresh_factory: bool = False, ) -> tuple[WorkflowInvocationKwargs, WorkflowInvocationKwargs]: @@ -484,9 +644,6 @@ def validate_values(value: Any) -> dict[str, Any]: values["options"] = validate_values(values["options"]) if _CLIENT_CONTROLS.intersection(values["options"]): raise ValueError("Workflow model options cannot override hosting, agent, or transport controls.") - model_options = validate_values(options or {}) - if _CLIENT_CONTROLS.intersection(model_options): - raise ValueError("Workflow model options cannot override hosting, agent, or transport controls.") identity = {"session_id": scope.session_id, "user_id": scope.user_id, "call_id": scope.call_id} global_client["additional_function_arguments"] = identity @@ -518,7 +675,7 @@ def validate_values(value: Any) -> dict[str, Any]: ): raise ValueError("store=false cannot disable an external workflow HistoryProvider.") values = specific_client.setdefault(executor_id, {}) - effective = {**global_model_options, **values.get("options", {}), **model_options} + effective = {**global_model_options, **values.get("options", {})} values["additional_function_arguments"] = identity if isinstance(agent_value, Agent): if stores: @@ -641,12 +798,31 @@ def validate_workflow_source(source: object) -> None: raise TypeError("A workflow factory must accept one request argument.") from exc +def _is_stateless_function_tool(value: object) -> bool: + if type(value) is not FunctionTool: + return False + return ( + inspect.isfunction(value.func) + and value.func.__closure__ is None + and value._instance is None # pyright: ignore[reportPrivateUsage] + and value.max_invocations is None + and value.max_invocation_exceptions is None + and ( + value.result_parser is None + or value.result_parser is SKIP_PARSING + or (inspect.isfunction(value.result_parser) and value.result_parser.__closure__ is None) + ) + ) + + class WorkflowResolver(Generic[RequestT]): """Reject reuse of mutable graphs and known request-bound resources. This is an ownership guard, not a clone operation or a proof that arbitrary - application globals are stateless. Factories must allocate their resources - without performing workflow or tool side effects. + application globals, function attributes, or default arguments are stateless. + Plain function tools without bound instances, closures, or lifetime limits may + be shared. Applications must keep those functions stateless. Factories must + allocate mutable resources without performing workflow or tool side effects. """ def __init__(self, source: WorkflowSource[RequestT]) -> None: @@ -691,13 +867,19 @@ def collect(graph: Workflow) -> None: resources.extend(agent.mcp_tools) tools = agent.default_options.get("tools", ()) if isinstance(tools, Sequence): - resources.extend(tool for tool in cast(Sequence[object], tools) if not isinstance(tool, Mapping)) + for tool in cast(Sequence[object], tools): + if isinstance(tool, Mapping) or _is_stateless_function_tool(tool): + continue + resources.append(tool) + if isinstance(tool, FunctionTool) and tool._instance is not None: # pyright: ignore[reportPrivateUsage] + resources.append(tool._instance) # pyright: ignore[reportPrivateUsage] unique_resources = {id(resource): resource for resource in resources} for identifier, resource in unique_resources.items(): previous = self._owned.get(id(resource)) if (previous is not None and previous() is resource) or self._strong_owned.get(identifier) is resource: raise RuntimeError( - "Native workflow requests cannot share workflows, executors, agents, clients, or providers. " + "Native workflow requests cannot share workflows, executors, agents, clients, providers, " + "or mutable tools. " "Use a request-aware factory that creates fresh instances." ) for identifier, resource in unique_resources.items(): diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py index 539287adc74..7eaac49224d 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py @@ -1,17 +1,17 @@ # Copyright (c) Microsoft. All rights reserved. +from __future__ import annotations import hashlib import json from abc import ABC, abstractmethod from datetime import datetime -from typing import Any, Generic, Protocol, TypeVar +from typing import Any, Generic, TypeVar from agent_framework import ( AgentSession, CheckpointID, CheckpointStorage, - Content, SessionStore, WorkflowCheckpoint, WorkflowCheckpointException, @@ -333,79 +333,6 @@ def get_store_for_scope( # endregion Checkpoint persistence -# region Function approval persistence - - -class FunctionApprovalStore(Protocol): - """Store for saving function approval requests.""" - - async def save_approval_request(self, approval_request_id: str, request: Content) -> None: - """Save a function approval request under the given ID.""" - ... - - async def load_approval_request(self, approval_request_id: str) -> Content: - """Load a function approval request by its ID.""" - ... - - -class FoundryFunctionApprovalStore: - """Function approval store backed by the `FoundryStateStore`.""" - - DEFAULT_ROOT_SCOPE = "function_approvals" - - def __init__( - self, - platform_context: FoundryAgentRequestContext, - *, - scope: FoundryRequestScope | None = None, - ) -> None: - self.platform_context = platform_context - self._scope = scope - - async def _get_store(self) -> FoundryStateStore: - return await FoundryStateStore.get_or_create( - _store_name(self.DEFAULT_ROOT_SCOPE, self._scope), - user_isolation=True, - ) - - async def save_approval_request(self, approval_request_id: str, request: Content) -> None: - store = await self._get_store() - async with store: - try: - await store.create_item(approval_request_id, request.to_dict(), call_id=self.platform_context.call_id) - except FoundryStorageConflictError as ex: - raise ValueError(f"Approval request with ID '{approval_request_id}' already exists.") from ex - - async def load_approval_request(self, approval_request_id: str) -> Content: - store = await self._get_store() - async with store: - item = await store.get_item(approval_request_id, call_id=self.platform_context.call_id) - if item is None: - raise KeyError(f"Approval request with ID '{approval_request_id}' does not exist.") - return Content.from_dict(item.value) - - -class FunctionApprovalStoreProvider(StoreProvider[FunctionApprovalStore]): - """Provide function approval store for the active hosting environment. - - This defaults to using the `FoundryFunctionApprovalStore` in all environments. - """ - - def get_store( - self, - *, - config: AgentConfig, - platform_context: FoundryAgentRequestContext, - ) -> FunctionApprovalStore: - """Get function approval store for the requested hosting environment.""" - return FoundryFunctionApprovalStore( - platform_context, - scope=_store_scope(config, platform_context), - ) - - -# endregion Function approval persistence - # region Agent session persistence @@ -442,6 +369,7 @@ async def _get_store(self) -> FoundryStateStore: ) async def get(self, session_id: str) -> AgentSession | None: + SessionStore.validate_session_id(session_id) store = await self._get_store() async with store: item = await store.get_item(session_id, call_id=self.platform_context.call_id) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py index d9a02501c56..63daf99ebff 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py @@ -15,6 +15,7 @@ import httpx2 as httpx from agent_framework import ( CachingSkillsSource, + Content, DeduplicatingSkillsSource, MCPSkillsSource, MCPStreamableHTTPTool, @@ -23,7 +24,7 @@ SkillsSourceContext, ) from agent_framework._telemetry import mark_feature_used -from agent_framework.exceptions import ToolExecutionException +from agent_framework.exceptions import ToolExecutionException, UserInputRequiredException from azure.ai.agentserver.core import get_request_context from mcp.types import Tool from typing_extensions import override @@ -34,7 +35,7 @@ from collections.abc import AsyncGenerator, Generator from datetime import timedelta - from agent_framework import Content, Skill + from agent_framework import Skill from agent_framework._mcp import MCPSpecificApproval from azure.core.credentials import AccessToken, TokenCredential from azure.core.credentials_async import AsyncTokenCredential @@ -412,9 +413,51 @@ async def _prepare_for_run(self, kwargs: Mapping[str, Any]) -> None: @override async def call_tool(self, tool_name: str, **kwargs: Any) -> str | list[Content]: + """Call a Toolbox tool and translate invocation-time OAuth consent when a hosted request scope is active. + + Inside a Foundry Responses host request, a gateway ``-32006`` consent error with an allowed HTTPS consent + link raises ``UserInputRequiredException`` carrying a terminal ``function_result`` and an + ``oauth_consent_request``. Later Toolbox calls in the same request return a not-executed result without + contacting the gateway. A rejected consent link raises a sanitized ``ToolExecutionException`` and marks the + request scope so the host fails the response. Outside a hosted request scope, errors propagate unchanged. + """ + from ._responses_shared import ( + OAUTH_CONSENT_NOT_EXECUTED_RESULT, + OAUTH_CONSENT_REQUIRED_RESULT, + ConsentError, + _current_toolbox_consent_scope, # pyright: ignore[reportPrivateUsage] + _is_allowed_oauth_consent_link, # pyright: ignore[reportPrivateUsage] + tool_call_consent_url_from_error, + ) + + scope = _current_toolbox_consent_scope() + if scope is not None and scope.blocks_toolbox_calls: + return OAUTH_CONSENT_NOT_EXECUTED_RESULT token = _strict_toolbox_header_provider.set(True) try: return await super().call_tool(tool_name, **kwargs) + except ToolExecutionException as exc: + if scope is None: + raise + consent_link = tool_call_consent_url_from_error(exc) + if consent_link is None: + raise + if not _is_allowed_oauth_consent_link(consent_link, scope.allowed_origins): + scope.consent_rejected = True + raise ToolExecutionException( + f"Toolbox '{self.name}' returned an OAuth consent link that is not allowed." + ) from None + scope.consents.append(ConsentError(name=self.name, consent_url=consent_link)) + raise UserInputRequiredException( + contents=[ + Content.from_function_result(call_id="", result=OAUTH_CONSENT_REQUIRED_RESULT), + Content.from_oauth_consent_request( + consent_link, + additional_properties={"server_label": self.name}, + ), + ], + message="Foundry Toolbox OAuth consent is required.", + ) from exc finally: _strict_toolbox_header_provider.reset(token) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py index ae822942767..34107962f8a 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py @@ -18,16 +18,12 @@ ChatResponse, ChatResponseUpdate, CheckpointStorage, - Content, Workflow, WorkflowCheckpoint, WorkflowEvent, WorkflowInvocationKwargs, WorkflowRunState, ) -from agent_framework._workflows._agent_executor import ( - _validate_computer_tool_result, # pyright: ignore[reportPrivateUsage] -) from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value from agent_framework._workflows._const import ( RESOLVED_WORKFLOW_RUN_KWARGS_KEY, @@ -35,9 +31,6 @@ WORKFLOW_RUN_KWARGS_KEY, ) from agent_framework._workflows._typing_utils import is_instance_of -from agent_framework._workflows._workflow import ( - _coerce_request_info_response, # pyright: ignore[reportPrivateUsage] -) from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext from azure.ai.agentserver.core.storage import ( FoundryStateStore, @@ -345,8 +338,9 @@ class HostedWorkflowRun: """A native turn with explicit validation, claim, pairing, commit, and abort boundaries. Protocol hosts own output encoding and lifecycle signals. Closing ``events`` - never commits a turn. A failed or cancelled claim is blocked rather than - releasing potentially consumed approvals or replaying external side effects. + never commits a turn. A claim that fails before the first superstep starts is + released, because no executor or reply handler has run. Later failures block + the lineage rather than replaying consumed replies or external side effects. """ def __init__( @@ -369,6 +363,8 @@ def __init__( self.binding: WorkflowBinding self._head: WorkflowHead | None = None self._head_etag: str | None = None + self._pre_claim_head: WorkflowHead | None = None + self._superstep_started = False self._record: WorkflowRecord | None = None self._record_etag: str | None = None self._storage: _CheckpointWrites | None = None @@ -652,49 +648,11 @@ def validate_turn(self, turn: WorkflowTurn[Any]) -> WorkflowTurn[Any]: return turn pending = self.pending_requests if turn.responses is not None: - if not self.stored or self._checkpoint is None: + # Core validates each reply against its pending request before the first superstep. + if not self.stored or self._checkpoint is None or not pending: raise ValueError("Workflow replies require a pending stored checkpoint.") - if set(turn.responses) != set(pending) or not pending: - raise ValueError("Workflow replies must answer exactly the complete pending request batch.") - replies: dict[str, Any] = {} - reply_ids: set[tuple[str, str]] = set() - for request_id, reply in turn.responses.items(): - event = pending[request_id] - try: - reply = _coerce_request_info_response(reply, event.response_type, request_id) - except (TypeError, ValueError) as exc: - raise ValueError("A workflow reply does not match its pending request type.") from exc - if isinstance(event.data, Content): - request = event.data - if isinstance(reply, Content): - identifier = reply.id if reply.type == "function_approval_response" else reply.call_id - if identifier is not None: - key = (reply.type, identifier) - if key in reply_ids: - raise ValueError("Duplicate workflow reply authority is not allowed.") - reply_ids.add(key) - if request.type == "function_approval_request" and ( - not isinstance(reply, Content) - or reply.type != "function_approval_response" - or reply.id != request.id - or type(reply.approved) is not bool - or reply.function_call is None - or request.function_call is None - or reply.function_call.to_dict() != request.function_call.to_dict() - ): - raise ValueError("The approval decision does not match its exact pending function call.") - if request.type == "function_call" and ( - not isinstance(reply, Content) - or reply.type != "function_result" - or reply.call_id != request.call_id - ): - raise ValueError("The function result does not match its pending call.") - if request.type == "computer_tool_call": - if not isinstance(reply, Content): - raise ValueError("A computer reply must be typed Content.") - _validate_computer_tool_result(request, reply) - replies[request_id] = reply - turn = replace(turn, responses=replies) + if not turn.responses: + raise ValueError("Workflow replies must answer at least one pending request.") else: if pending: raise ValueError("This workflow is awaiting replies; do not replace them with new input.") @@ -715,6 +673,7 @@ async def claim(self) -> None: self._claimed = True return owner = secrets.token_hex(16) + self._pre_claim_head = self._head head = replace( self._head or WorkflowHead(self.binding.lineage_id, self.binding.conversation_id), response_id=self.binding.response_id, @@ -730,7 +689,6 @@ async def claim(self) -> None: ) self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) self._record = record - await self._reclaim_completed_ancestors() async def assert_claim(self) -> None: if not self._claimed: @@ -776,6 +734,8 @@ async def events( and data.continuation_token is not None ): raise RuntimeError("Native workflows cannot checkpoint unfinished provider background output.") + if event.type == "superstep_started": + self._superstep_started = True if event.type == "superstep_completed": validate_workflow_provider_state(self.workflow) if event.type == "request_info" and not self.stored: @@ -871,6 +831,7 @@ async def commit(self, snapshot: Mapping[str, Any] | None = None) -> None: self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) self._head = head self._claimed = False + await self._reclaim_completed_ancestors() async def _reclaim_completed_ancestors(self) -> None: """Best-effort reclaim after the new head is durably committed. @@ -911,10 +872,23 @@ async def _reclaim_completed_ancestors(self) -> None: logger.exception("Failed to reclaim superseded native workflow state; a later commit can retry cleanup") async def abort(self) -> None: - """Block interrupted/failed authority instead of permitting unsafe reply or effect replay.""" + """Release a claim that never started a superstep; otherwise block the lineage against unsafe replay.""" if not self.stored or not self._claimed or self.already_completed: return await self.assert_claim() + if ( + not self._superstep_started + and not self.recovery + and self._pre_claim_head is not None + and self._record_etag is not None + ): + # Core rejects invalid replies before any executor runs, so the head and checkpoint stay usable. + self._head_etag = await self.store.save_head(self._pre_claim_head, expected_etag=self._head_etag) + self._head = self._pre_claim_head + await self.store.delete_response(self.binding.response_id, expected_etag=self._record_etag) + self._record = self._record_etag = None + self._claimed = False + return if self._record is not None: record = replace(self._record, status="blocked") self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/agent_approval_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/agent_approval_helpers.py new file mode 100644 index 00000000000..75e42899a6b --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/agent_approval_helpers.py @@ -0,0 +1,115 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Real-Agent boundaries for hosted approval round trips.""" + +from __future__ import annotations + +import json +from collections.abc import AsyncIterator, Awaitable, Mapping, Sequence +from pathlib import Path +from typing import Any + +import httpx2 as httpx +from agent_framework import ( + Agent, + BaseChatClient, + ChatResponse, + ChatResponseUpdate, + Content, + FunctionInvocationLayer, + FunctionTool, + Message, + ResponseStream, + tool, +) + +from agent_framework_foundry_hosting import ResponsesHostServer + +from .approval_helpers import SqliteSessionStore +from .toolbox_approval_helpers import FixedSessionStoreProvider + + +class BatchApprovalClient(FunctionInvocationLayer[Any], BaseChatClient[Any]): + """Call the protected tool for each name after user text; answer with text otherwise.""" + + def __init__(self, names: Sequence[str]) -> None: + super().__init__() + self.names = list(names) + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + last = messages[-1] + contents = ( + [ + Content.from_function_call(call_id=f"call-{name}", name="protected", arguments={"name": name}) + for name in self.names + ] + if last.role == "user" and any(content.type == "text" for content in last.contents) + else [Content.from_text("done")] + ) + + async def updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate(role="assistant", contents=contents) + + async def complete() -> ChatResponse: + return ChatResponse(messages=[Message("assistant", contents=contents)]) + + return ResponseStream(updates(), finalizer=ChatResponse.from_updates) if stream else complete() + + +def protected_tool(executed: list[str]) -> FunctionTool: + @tool(approval_mode="always_require") + def protected(name: str) -> str: + """Run a protected action.""" + executed.append(name) + return f"ran {name}" + + return protected + + +def approval_host(tmp_path: Path, executed: list[str], names: Sequence[str] = ("a", "b")) -> ResponsesHostServer: + return ResponsesHostServer( + agent=Agent(client=BatchApprovalClient(names), tools=[protected_tool(executed)]), + agent_session_store_provider=FixedSessionStoreProvider(SqliteSessionStore(tmp_path / "sessions.db")), + ) + + +async def post_turn(client: httpx.AsyncClient, payload: dict[str, Any], *, stream: bool) -> dict[str, Any]: + """Post one turn and return its terminal response body.""" + response = await client.post("/responses", json={"model": "test-model", **payload, "stream": stream}) + assert response.status_code == 200, response.text + if not stream: + return response.json() + events = [ + json.loads(line[6:]) + for line in response.text.splitlines() + if line.startswith("data: ") and line != "data: [DONE]" + ] + return next( + event["response"] + for event in events + if event["type"] in ("response.completed", "response.failed", "response.incomplete") + ) + + +def approval_ids(body: dict[str, Any]) -> dict[str, str]: + """Map each requested tool argument name to its wire approval id.""" + return { + json.loads(item["arguments"])["name"]: item["id"] + for item in body["output"] + if item["type"] == "mcp_approval_request" + } + + +def approval_reply(approval_request_id: str, approve: bool) -> dict[str, Any]: + return {"type": "mcp_approval_response", "approval_request_id": approval_request_id, "approve": approve} + + +def tool_outputs(body: dict[str, Any]) -> list[str]: + return [item["output"] for item in body["output"] if item["type"] == "function_call_output"] diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/approval_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/approval_helpers.py new file mode 100644 index 00000000000..738fab71808 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/approval_helpers.py @@ -0,0 +1,72 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Durable session-store boundaries for hosted approval regression tests.""" + +from __future__ import annotations + +import asyncio +import json +import sqlite3 +from contextlib import closing +from pathlib import Path + +from agent_framework import AgentSession, Content, SessionStore + +from agent_framework_foundry_hosting._approvals import SessionApprovalState + + +class SqliteSessionStore(SessionStore): + """Persist sessions in a real database shared by independent processes.""" + + def __init__(self, path: Path) -> None: + self.path = path + self.write_count = 0 + with closing(sqlite3.connect(path)) as connection, connection: + connection.execute("CREATE TABLE IF NOT EXISTS sessions (key TEXT PRIMARY KEY, payload TEXT NOT NULL)") + + def _read(self, session_id: str) -> AgentSession | None: + with closing(sqlite3.connect(self.path)) as connection: + row = connection.execute("SELECT payload FROM sessions WHERE key = ?", (session_id,)).fetchone() + return AgentSession.from_dict(json.loads(row[0])) if row else None + + async def get(self, session_id: str) -> AgentSession | None: + return await asyncio.to_thread(self._read, session_id) + + def _write(self, session_id: str, payload: str) -> None: + with closing(sqlite3.connect(self.path)) as connection, connection: + connection.execute( + "INSERT INTO sessions VALUES (?, ?) ON CONFLICT(key) DO UPDATE SET payload = excluded.payload", + (session_id, payload), + ) + + async def set(self, session_id: str, session: AgentSession) -> None: + await asyncio.to_thread(self._write, session_id, json.dumps(session.to_dict())) + self.write_count += 1 + + async def delete(self, session_id: str) -> None: + await asyncio.to_thread(self._delete, session_id) + + def _delete(self, session_id: str) -> None: + with closing(sqlite3.connect(self.path)) as connection, connection: + connection.execute("DELETE FROM sessions WHERE key = ?", (session_id,)) + + +def approval_request(index: int = 0) -> Content: + call = Content.from_function_call( + call_id=f"call-{index}", + name="protected", + arguments={"index": index}, + hosted=True, + additional_properties={"provider": {"type": "message", "opaque_id": "original"}}, + ) + return Content.from_function_approval_request(f"approval-{index}", call) + + +async def pending_session(store: SqliteSessionStore, count: int = 2) -> AgentSession: + session = AgentSession(session_id="original-session") + state = SessionApprovalState(session) + for index in range(count): + await state.save_approval_request(f"wire-{index}", approval_request(index)) + await store.set("response-alias", session) + await store.set("conversation-alias", session) + return session diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/approval_translation_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/approval_translation_helpers.py new file mode 100644 index 00000000000..d737f259a3e --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/approval_translation_helpers.py @@ -0,0 +1,37 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Helpers for Responses approval translation tests.""" + +from __future__ import annotations + +from collections.abc import AsyncIterator, Mapping +from typing import Any + +from agent_framework import Content + + +def _approval_request(request_id: str = "orig-1") -> Content: + return Content.from_function_approval_request( + id=request_id, + function_call=Content.from_function_call("call-1", "dangerous", arguments='{"x": 1}'), + ) + + +class _OrderedRecorder: + def __init__(self, log: list[str]) -> None: + self._log = log + self.saved: list[tuple[str, Content]] = [] + + async def save_approval_request(self, approval_request_id: str, request: Content) -> None: + self._log.append(f"save:{approval_request_id}") + self.saved.append((approval_request_id, request)) + + +def _event_type(event: Any) -> str: + event_type = event.get("type") if isinstance(event, Mapping) else event.type # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] + assert isinstance(event_type, str) + return event_type + + +async def _events_without_id(*_: Any, **__: Any) -> AsyncIterator[dict[str, Any]]: + yield {"type": "response.output_item.added", "item": {"type": "mcp_approval_request"}} diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/background_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/background_helpers.py new file mode 100644 index 00000000000..27cc68a7495 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/background_helpers.py @@ -0,0 +1,68 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Real agents and middleware for provider-background boundary tests.""" + +from __future__ import annotations + +from collections.abc import Awaitable, Callable +from typing import Any + +from agent_framework import ( + Agent, + AgentContext, + ChatContext, + ChatMiddlewareLayer, + ChatMiddlewareTypes, + RawAgent, + Workflow, + WorkflowBuilder, +) +from starlette.requests import Request + +from .options_helpers import OptionsClient + + +class LayeredOptionsClient(ChatMiddlewareLayer, OptionsClient): + pass + + +class BackgroundCallbacks: + def __init__( + self, + *, + source: str, + location: str, + injected: dict[str, Any], + raw: bool = False, + ) -> None: + self.client = LayeredOptionsClient() + self.source = source + self.location = location + self.injected = injected + middleware: list[ChatMiddlewareTypes] = [self.inject_chat] if source == "chat" else [] + self.client.chat_middleware = middleware + self.agent = ( + RawAgent(client=self.client) + if raw + else Agent(client=self.client, middleware=[self.inject_agent] if source == "agent" else None) + ) + + async def inject_agent(self, context: AgentContext, call_next: Callable[[], Awaitable[None]]) -> None: + if self.location == "options": + context.options = {**(context.options or {}), **self.injected} + else: + context.client_kwargs.update(self.injected) + await call_next() + + async def inject_chat(self, context: ChatContext, call_next: Callable[[], Awaitable[None]]) -> None: + if self.location == "options": + context.options = {**(context.options or {}), **self.injected} + else: + context.kwargs.update(self.injected) + await call_next() + + def factory(self, *, request: Request) -> Agent[Any] | RawAgent[Any]: + return self.agent + + def workflow(self, request: Request) -> Workflow: + return WorkflowBuilder(start_executor=self.agent).build() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/custom_storage_sample_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/custom_storage_sample_helpers.py new file mode 100644 index 00000000000..f2ce84216ff --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/custom_storage_sample_helpers.py @@ -0,0 +1,88 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Offline storage boundaries for the custom session-storage sample.""" + +from __future__ import annotations + +from copy import deepcopy +from types import ModuleType +from typing import Any + +from agent_framework import AgentSession +from azure.core import MatchConditions +from azure.cosmos.exceptions import CosmosHttpResponseError, CosmosResourceNotFoundError + +from agent_framework_foundry_hosting import FoundryRequestScope + + +def make_session(value: str) -> AgentSession: + session = AgentSession(session_id="inner-maf-session") + session.state["value"] = value + return session + + +def http_error(status: int) -> CosmosHttpResponseError: + if status == 404: + return CosmosResourceNotFoundError(status_code=404, message="missing") + return CosmosHttpResponseError(status_code=status, message=f"status {status}") + + +class FakeContainer: + """Emulate Cosmos create-only and exact-ETag operations in one event loop.""" + + def __init__(self) -> None: + self.items: dict[tuple[str, str], dict[str, Any]] = {} + self._version = 0 + self.create_error: int | None = None + self.calls: list[tuple[str, dict[str, Any]]] = [] + + def _stamp(self, body: dict[str, Any]) -> dict[str, Any]: + self._version += 1 + stored = {**deepcopy(body), "_etag": f'"{self._version}"'} + self.items[body["scope_key"], body["id"]] = stored + return deepcopy(stored) + + async def read_item(self, *, item: str, partition_key: str) -> dict[str, Any]: + stored = self.items.get((partition_key, item)) + if stored is None: + raise http_error(404) + return deepcopy(stored) + + async def create_item(self, *, body: dict[str, Any]) -> dict[str, Any]: + self.calls.append(("create", {})) + if self.create_error is not None: + raise http_error(self.create_error) + if (body["scope_key"], body["id"]) in self.items: + raise http_error(409) + return self._stamp(body) + + async def replace_item( + self, *, item: str, body: dict[str, Any], etag: str, match_condition: MatchConditions + ) -> dict[str, Any]: + self.calls.append(("replace", {"etag": etag, "match_condition": match_condition})) + assert match_condition == MatchConditions.IfNotModified + stored = self.items.get((body["scope_key"], item)) + if stored is None: + raise http_error(404) + if etag != stored["_etag"]: + raise http_error(412) + return self._stamp(body) + + async def delete_item(self, *, item: str, partition_key: str, etag: str, match_condition: MatchConditions) -> None: + self.calls.append(("delete", {"etag": etag, "match_condition": match_condition})) + assert match_condition == MatchConditions.IfNotModified + stored = self.items.get((partition_key, item)) + if stored is None: + raise http_error(404) + if etag != stored["_etag"]: + raise http_error(412) + del self.items[partition_key, item] + + +def cosmos_store(sample: ModuleType, container: FakeContainer) -> Any: + scope = FoundryRequestScope(session_id="sandbox", user_id="user", call_id="call", is_hosted=True) + return sample.CosmosSessionStore(container=container, scope=scope) + + +def local_store(sample: ModuleType, snapshots: dict[Any, Any], scope_key: str = "scope") -> Any: + return sample.LocalSessionStore(snapshots, scope_key=scope_key) diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/options_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/options_helpers.py new file mode 100644 index 00000000000..ccb067bcb8c --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/options_helpers.py @@ -0,0 +1,85 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Offline clients and callbacks for host option-boundary tests.""" + +from __future__ import annotations + +import asyncio +from collections.abc import AsyncIterator, Awaitable, Mapping, Sequence +from typing import Any, cast + +from agent_framework import ( + Agent, + AgentContext, + BaseChatClient, + ChatOptions, + ChatResponse, + ChatResponseUpdate, + Content, + Message, + ResponseStream, + Workflow, + WorkflowBuilder, +) +from starlette.requests import Request + + +class OptionsClient(BaseChatClient[ChatOptions]): + STORES_BY_DEFAULT = False + + def __init__(self, *, delay: float = 0) -> None: + super().__init__() + self.calls: list[dict[str, Any]] = [] + self.delay = delay + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + self.calls.append(dict(options)) + return ResponseStream(self.updates(), finalizer=ChatResponse.from_updates) if stream else self.complete() + + async def updates(self) -> AsyncIterator[ChatResponseUpdate]: + if self.delay: + await asyncio.sleep(self.delay) + yield ChatResponseUpdate(role="assistant", contents=[Content.from_text("ok")]) + + async def complete(self) -> ChatResponse: + return ChatResponse(messages=[Message("assistant", ["ok"])]) + + +class OptionCallbacks: + def __init__(self, *, defaults: dict[str, Any] | None = None, injected: dict[str, Any] | None = None) -> None: + self.client = OptionsClient() + self.defaults = cast(ChatOptions, defaults or {}) + self.injected = injected or {} + self.events: list[str] = [] + + def factory(self, *, request: Request) -> Agent: + self.events.append("factory") + return Agent(client=self.client, default_options=self.defaults) + + def prepare(self, request: Request, options: dict[str, Any]) -> dict[str, Any]: + self.events.append("hook") + options.pop("model", None) + return options + + async def prepare_async(self, request: Request, options: dict[str, Any]) -> dict[str, Any]: + return self.prepare(request, options) + + async def inject(self, context: AgentContext, call_next: Any) -> None: + context.options = {**(context.options or {}), **self.injected} + await call_next() + + def workflow(self, request: Request) -> Workflow: + self.events.append("factory") + agent = Agent( + client=self.client, + default_options=self.defaults, + middleware=[self.inject] if self.injected else None, + ) + return WorkflowBuilder(start_executor=agent).build() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/responses_host_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/responses_host_helpers.py new file mode 100644 index 00000000000..dfecee7b2cf --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/responses_host_helpers.py @@ -0,0 +1,249 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Controlled provider and storage boundaries for Responses host integration tests.""" + +from __future__ import annotations + +import asyncio +from collections.abc import AsyncIterator, Awaitable, Callable, Mapping, Sequence +from typing import Any + +from agent_framework import ( + AgentContext, + AgentMiddleware, + AgentResponseUpdate, + AgentSession, + BaseChatClient, + ChatContext, + ChatMiddleware, + ChatMiddlewareLayer, + ChatResponse, + ChatResponseUpdate, + Content, + FunctionInvocationLayer, + Message, + ResponseStream, + SessionStore, +) +from agent_framework.openai import OpenAIContinuationToken + + +class BackgroundAgentMiddleware(AgentMiddleware): + def __init__(self, *, nested: bool, value: bool = True, in_kwargs: bool = False, on_poll: bool = False) -> None: + self.nested = nested + self.value = value + self.in_kwargs = in_kwargs + self.on_poll = on_poll + + async def process(self, context: AgentContext, call_next: Callable[[], Awaitable[None]]) -> None: + if not self.on_poll or (context.options or {}).get("continuation_token") is not None: + override = {"extra_body": {"background": self.value}} if self.nested else {"background": self.value} + if self.in_kwargs: + context.client_kwargs.update(override) + else: + context.options = {**(context.options or {}), **override} + await call_next() + + +class BackgroundChatMiddleware(ChatMiddleware): + def __init__(self, *, nested: bool, value: bool = True, in_kwargs: bool = False, on_poll: bool = False) -> None: + self.nested = nested + self.value = value + self.in_kwargs = in_kwargs + self.on_poll = on_poll + + async def process(self, context: ChatContext, call_next: Callable[[], Awaitable[None]]) -> None: + if not self.on_poll or (context.options or {}).get("continuation_token") is not None: + override = {"extra_body": {"background": self.value}} if self.nested else {"background": self.value} + if self.in_kwargs: + context.kwargs.update(override) + else: + context.options = {**(context.options or {}), **override} + await call_next() + + +class FailingAgentMiddleware(AgentMiddleware): + async def process(self, context: AgentContext, call_next: Callable[[], Awaitable[None]]) -> None: + raise RuntimeError("Application middleware failed.") + + +class BackgroundPollingClient(ChatMiddlewareLayer[Any], BaseChatClient[Any]): + STORES_BY_DEFAULT = True + + def __init__(self) -> None: + super().__init__() + self.calls: list[dict[str, Any]] = [] + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + del messages, kwargs + assert not stream + self.calls.append(dict(options)) + + async def response() -> ChatResponse: + if len(self.calls) == 1: + return ChatResponse( + conversation_id="provider-conversation", + continuation_token=OpenAIContinuationToken(response_id="provider-token"), + ) + return ChatResponse(messages=[Message("assistant", "done")], conversation_id="provider-conversation") + + return response() + + +class ApprovalPublishingClient(ChatMiddlewareLayer[Any], BaseChatClient[Any]): + STORES_BY_DEFAULT = False + + def __init__(self, approval: Content) -> None: + super().__init__() + self.approval = approval + self.calls = 0 + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + del messages, options, kwargs + assert stream + self.calls += 1 + + async def updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate(role="assistant", contents=[self.approval]) + + return ResponseStream(updates(), finalizer=ChatResponse.from_updates) + + +class UnlayeredClient(BaseChatClient[Any]): + STORES_BY_DEFAULT = True + + def __init__(self) -> None: + super().__init__() + self.kwargs: list[dict[str, Any]] = [] + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + del messages, options + self.kwargs.append(dict(kwargs)) + + async def updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate(role="assistant", contents=[Content.from_text("done")], conversation_id="service") + + async def response() -> ChatResponse: + return ChatResponse(messages=[Message("assistant", "done")], conversation_id="service") + + return ResponseStream(updates(), finalizer=ChatResponse.from_updates) if stream else response() + + +class GatedRequestRead: + def __init__(self) -> None: + self.started = asyncio.Event() + self.stopped = asyncio.Event() + + async def load(self) -> list[Any]: + self.started.set() + try: + await asyncio.Event().wait() + finally: + self.stopped.set() + return [] + + +class GatedSessionLoadStore(SessionStore): + def __init__(self, request_started: asyncio.Event) -> None: + super().__init__() + self.request_started = request_started + self.started = asyncio.Event() + self.stopped = asyncio.Event() + + async def get(self, session_id: str) -> AgentSession | None: + del session_id + await self.request_started.wait() + self.started.set() + try: + await asyncio.Event().wait() + finally: + self.stopped.set() + return None + + +class GatedAgentStream: + def __init__(self) -> None: + self.started = asyncio.Event() + self.release = asyncio.Event() + self.stopped = asyncio.Event() + + async def updates(self) -> AsyncIterator[AgentResponseUpdate]: + self.started.set() + try: + await self.release.wait() + yield AgentResponseUpdate(role="assistant", contents=[Content.from_text("too late")]) + finally: + self.stopped.set() + + +async def collect_host_events(source: AsyncIterator[Any], events: list[Any]) -> None: + async for event in source: + events.append(event) + + +class StatefulConsentClient(FunctionInvocationLayer[Any], ChatMiddlewareLayer[Any], BaseChatClient[Any]): + """Mimic a service-backed conversation with one settled and one consent batch.""" + + STORES_BY_DEFAULT = True + + def __init__(self, tool_names: Sequence[str]) -> None: + super().__init__() + self.tool_names = list(tool_names) + self.calls: list[list[Message]] = [] + self.options: list[dict[str, Any]] = [] + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + del kwargs + self.calls.append(list(messages)) + self.options.append(dict(options)) + if len(self.calls) == 1: + contents = [Content.from_function_call(call_id="old-0", name=self.tool_names[0], arguments="{}")] + elif len(self.calls) == 2: + contents = [ + Content.from_function_call(call_id=f"pending-{index}", name=name, arguments="{}") + for index, name in enumerate(self.tool_names) + ] + else: + contents = [Content.from_text("done")] + + async def updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate( + role="assistant", contents=contents, conversation_id="provider-conversation", response_id="provider" + ) + + async def complete() -> ChatResponse: + return ChatResponse( + messages=[Message("assistant", contents=contents)], + conversation_id="provider-conversation", + response_id="provider", + ) + + return ResponseStream(updates(), finalizer=ChatResponse.from_updates) if stream else complete() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_agent_approvals.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_agent_approvals.py new file mode 100644 index 00000000000..0a48e38b4e3 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_agent_approvals.py @@ -0,0 +1,126 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Hosted agents rely on Core pending approvals in the same AgentSession.""" + +from __future__ import annotations + +from pathlib import Path + +import httpx2 as httpx +import pytest +from azure.ai.agentserver.responses._id_generator import IdGenerator + +from .agent_approval_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + approval_host, + approval_ids, + approval_reply, + post_turn, + tool_outputs, +) + +REJECTED = "Error: Tool call invocation was rejected by user." + +pytestmark = pytest.mark.parametrize("stream", [False, True]) + + +async def test_mixed_batch_runs_approved_call_and_rejects_the_other(stream: bool, tmp_path: Path) -> None: + executed: list[str] = [] + host = approval_host(tmp_path, executed) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + first = await post_turn(client, {"input": "go"}, stream=stream) + ids = approval_ids(first) + assert set(ids) == {"a", "b"} + assert executed == [] + + second = await post_turn( + client, + { + "input": [approval_reply(ids["a"], True), approval_reply(ids["b"], False)], + "previous_response_id": first["id"], + }, + stream=stream, + ) + + assert second["status"] == "completed" + assert sorted(tool_outputs(second)) == sorted(["ran a", REJECTED]) + assert executed == ["a"] + + +async def test_partial_batch_keeps_the_unanswered_request_pending(stream: bool, tmp_path: Path) -> None: + executed: list[str] = [] + host = approval_host(tmp_path, executed) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + first = await post_turn(client, {"input": "go"}, stream=stream) + ids = approval_ids(first) + + partial = await post_turn( + client, + {"input": [approval_reply(ids["a"], True)], "previous_response_id": first["id"]}, + stream=stream, + ) + assert partial["status"] == "completed" + assert tool_outputs(partial) == [] + assert executed == [] + remaining = approval_ids(partial) + assert list(remaining) == ["b"] + + final = await post_turn( + client, + {"input": [approval_reply(remaining["b"], False)], "previous_response_id": partial["id"]}, + stream=stream, + ) + + assert sorted(tool_outputs(final)) == sorted(["ran a", REJECTED]) + assert executed == ["a"] + + +async def test_replayed_approval_is_not_executed_again(stream: bool, tmp_path: Path) -> None: + executed: list[str] = [] + host = approval_host(tmp_path, executed, names=("a",)) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + first = await post_turn(client, {"input": "go"}, stream=stream) + reply = {"input": [approval_reply(approval_ids(first)["a"], True)]} + second = await post_turn(client, {**reply, "previous_response_id": first["id"]}, stream=stream) + assert tool_outputs(second) == ["ran a"] + + replay = await post_turn(client, {**reply, "previous_response_id": second["id"]}, stream=stream) + + assert tool_outputs(replay) == [] + assert executed == ["a"] + + +async def test_conversation_continuation_runs_approved_calls(stream: bool, tmp_path: Path) -> None: + executed: list[str] = [] + host = approval_host(tmp_path, executed) + conversation = IdGenerator.new_id("conv") + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + first = await post_turn(client, {"input": "go", "conversation": conversation}, stream=stream) + ids = approval_ids(first) + + second = await post_turn( + client, + { + "input": [approval_reply(ids["a"], True), approval_reply(ids["b"], True)], + "conversation": conversation, + }, + stream=stream, + ) + + assert sorted(tool_outputs(second)) == ["ran a", "ran b"] + assert sorted(executed) == ["a", "b"] + + +async def test_fabricated_approval_is_not_executed(stream: bool, tmp_path: Path) -> None: + executed: list[str] = [] + host = approval_host(tmp_path, executed) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + first = await post_turn(client, {"input": "go"}, stream=stream) + forged = await post_turn( + client, + {"input": [approval_reply("mcpr_forged", True)], "previous_response_id": first["id"]}, + stream=stream, + ) + + assert forged["status"] == "failed" + assert tool_outputs(forged) == [] + assert executed == [] diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_approval_translation.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_approval_translation.py new file mode 100644 index 00000000000..5dfc30afc10 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_approval_translation.py @@ -0,0 +1,223 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Responses approval translation and output staging.""" + +from __future__ import annotations + +from copy import deepcopy +from pathlib import Path +from typing import Any + +import pytest +from agent_framework import AgentSession +from azure.ai.agentserver.responses.aio import ResponseEventStream +from azure.ai.agentserver.responses.models import ( + Item, + ItemMcpApprovalRequest, + MCPApprovalResponse, + OutputItem, + OutputItemMcpApprovalRequest, + OutputItemMcpApprovalResponseResource, +) + +from agent_framework_foundry_hosting._approvals import SessionApprovalState +from agent_framework_foundry_hosting._responses_shared import ( + _items_to_messages, + _output_items_to_messages, + _OutputItemTracker, +) + +from .approval_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + SqliteSessionStore, + approval_request, + pending_session, +) +from .approval_translation_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + _approval_request, + _event_type, + _events_without_id, + _OrderedRecorder, +) + + +@pytest.mark.parametrize("approved", [True, False]) +async def test_history_and_input_conversions_only_read_approval_state(tmp_path: Path, approved: bool) -> None: + store = SqliteSessionStore(tmp_path / "sessions.db") + session = await pending_session(store) + state = SessionApprovalState(session) + session_before = deepcopy(session.to_dict()) + writes_before = store.write_count + stored_before: dict[str, dict[str, Any]] = {} + for key in ("response-alias", "conversation-alias"): + stored = await store.get(key) + assert stored is not None + stored_before[key] = stored.to_dict() + + output_items: list[OutputItem] = [ + OutputItemMcpApprovalRequest({ + "type": "mcp_approval_request", + "id": "wire-0", + "server_label": "untrusted", + "name": "untrusted", + "arguments": '{"index": 999}', + }), + OutputItemMcpApprovalResponseResource({ + "type": "mcp_approval_response", + "id": "resp-1", + "approval_request_id": "wire-0", + "approve": approved, + }), + ] + input_items: list[Item] = [ + ItemMcpApprovalRequest({ + "type": "mcp_approval_request", + "id": "wire-0", + "server_label": "untrusted", + "name": "untrusted", + "arguments": '{"index": 999}', + }), + MCPApprovalResponse({ + "type": "mcp_approval_response", + "approval_request_id": "wire-0", + "approve": approved, + }), + ] + + output_messages = await _output_items_to_messages(output_items, approval_storage=state) + input_messages = await _items_to_messages(input_items, approval_storage=state) + + for messages in (output_messages, input_messages): + assert [message.role for message in messages] == ["assistant", "user"] + request = messages[0].contents[0] + response = messages[1].contents[0] + assert request == approval_request() + assert response.type == "function_approval_response" + assert response.id == "approval-0" + assert response.approved is approved + assert response.function_call == request.function_call + + assert session.to_dict() == session_before + assert store.write_count == writes_before + for key, expected in stored_before.items(): + stored = await store.get(key) + assert stored is not None + assert stored.to_dict() == expected + + +async def test_unknown_input_approval_is_dropped_without_mutating_session(tmp_path: Path) -> None: + store = SqliteSessionStore(tmp_path / "sessions.db") + session = await pending_session(store) + state = SessionApprovalState(session) + session_before = deepcopy(session.to_dict()) + writes_before = store.write_count + items: list[Item] = [ + MCPApprovalResponse({"type": "mcp_approval_response", "approval_request_id": "wire-0", "approve": True}), + MCPApprovalResponse({"type": "mcp_approval_response", "approval_request_id": "missing", "approve": False}), + ] + + messages = await _items_to_messages(items, approval_storage=state) + + assert len(messages) == 1 + assert messages[0].contents[0].id == "approval-0" + assert session.to_dict() == session_before + assert store.write_count == writes_before + + +@pytest.mark.parametrize( + "item", + [ + OutputItemMcpApprovalRequest({ + "type": "mcp_approval_request", + "id": "missing", + "server_label": "srv", + "name": "dangerous", + "arguments": "{}", + }), + OutputItemMcpApprovalResponseResource({ + "type": "mcp_approval_response", + "id": "resp-1", + "approval_request_id": "missing", + "approve": True, + }), + ], + ids=["request", "response"], +) +async def test_history_conversion_drops_unknown_approval_ids(item: OutputItem) -> None: + state = SessionApprovalState(AgentSession()) + + assert await _output_items_to_messages([item], approval_storage=state) == [] + + +@pytest.mark.parametrize( + ("input_item", "output_item"), + [ + ( + ItemMcpApprovalRequest({ + "type": "mcp_approval_request", + "id": "wire-1", + "server_label": "srv", + "name": "dangerous", + "arguments": "{}", + }), + OutputItemMcpApprovalRequest({ + "type": "mcp_approval_request", + "id": "wire-1", + "server_label": "srv", + "name": "dangerous", + "arguments": "{}", + }), + ), + ( + MCPApprovalResponse({"type": "mcp_approval_response", "approval_request_id": "wire-1", "approve": True}), + OutputItemMcpApprovalResponseResource({ + "type": "mcp_approval_response", + "id": "resp-1", + "approval_request_id": "wire-1", + "approve": True, + }), + ), + ], + ids=["request", "response"], +) +async def test_approval_conversion_requires_session_state(input_item: Item, output_item: OutputItem) -> None: + with pytest.raises(ValueError, match="ApprovalStorage is required"): + await _items_to_messages([input_item]) + with pytest.raises(ValueError, match="ApprovalStorage is required"): + await _output_items_to_messages([output_item]) + + +async def test_tracker_records_approval_before_yielding_its_events() -> None: + stream = ResponseEventStream(response_id="resp_approval") + stream.emit_created() + stream.emit_in_progress() + tracker = _OutputItemTracker(stream) + log: list[str] = [] + recorder = _OrderedRecorder(log) + request = _approval_request() + + async for event in tracker.handle(request, approval_storage=recorder): + log.append(_event_type(event)) + log.extend(_event_type(event) for event in tracker.close()) + + assert len(recorder.saved) == 1 + wire_id, saved = recorder.saved[0] + assert wire_id + assert saved is request + assert log[0] == f"save:{wire_id}" + assert "response.output_item.added" in log[1:] + + +async def test_tracker_fails_explicitly_without_generated_approval_id(monkeypatch: pytest.MonkeyPatch) -> None: + stream = ResponseEventStream(response_id="resp_approval") + stream.emit_created() + stream.emit_in_progress() + + monkeypatch.setattr(stream, "output_item_mcp_approval_request", _events_without_id) + tracker = _OutputItemTracker(stream) + recorder = _OrderedRecorder([]) + + with pytest.raises(RuntimeError, match="approval request ID could not be extracted"): + async for _ in tracker.handle(_approval_request(), approval_storage=recorder): + pass + + assert recorder.saved == [] diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_background_middleware.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_background_middleware.py new file mode 100644 index 00000000000..e78ac4f1449 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_background_middleware.py @@ -0,0 +1,85 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Check provider-background validators and native-workflow middleware boundaries.""" + +from __future__ import annotations + +from typing import Any + +import httpx +import pytest + +from agent_framework_foundry_hosting import InvocationsHostServer, InvocationsWorkflowHostServer, WorkflowTurn +from agent_framework_foundry_hosting._shared import ( + validate_provider_background_options, +) + +from .background_helpers import BackgroundCallbacks + + +@pytest.mark.parametrize("provider_background", [False, True]) +@pytest.mark.parametrize("background", [None, False, True, 0, "true"]) +def test_pure_background_validator(provider_background: bool, background: Any) -> None: + options = {"background": background, "temperature": 0.4} + original = options.copy() + if background is not None and not isinstance(background, bool): + with pytest.raises(TypeError, match="boolean"): + validate_provider_background_options(options, provider_background=provider_background) + elif background is True and not provider_background: + with pytest.raises(ValueError, match="outer background"): + validate_provider_background_options(options, provider_background=provider_background) + else: + validate_provider_background_options(options, provider_background=provider_background) + assert options == original + + +@pytest.mark.parametrize("provider_background", [False, True]) +@pytest.mark.parametrize("background", [None, False, True]) +def test_pure_validator_rejects_all_nested_background(provider_background: bool, background: Any) -> None: + with pytest.raises(ValueError, match="host-controlled"): + validate_provider_background_options( + {"extra_body": {"background": background}}, provider_background=provider_background + ) + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("shape", ["flat", "options"]) +async def test_native_raw_agent_rejects_run_level_middleware(stream: bool, shape: str) -> None: + callbacks = BackgroundCallbacks(source="none", location="options", injected={"background": True}, raw=True) + client_kwargs: dict[str, Any] = {"middleware": [callbacks.inject_chat]} + if shape == "options": + client_kwargs = {"options": client_kwargs} + host = InvocationsWorkflowHostServer( + callbacks.workflow, + parse_input=lambda request: WorkflowTurn(input="hi", stream=stream, client_kwargs=client_kwargs), + ) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/invocations?agent_session_id=raw-middleware", json={"message": "hi"}) + assert response.status_code == (200 if stream else 400) + assert "cannot override hosting" in response.text + if stream: + assert "event: error" in response.text + assert callbacks.client.calls == [] + + +@pytest.mark.parametrize("workflow", [False, True]) +@pytest.mark.parametrize("raw", [False, True]) +@pytest.mark.parametrize("stream", [False, True]) +async def test_supported_middleware_preserves_foreground_generation(workflow: bool, raw: bool, stream: bool) -> None: + callbacks = BackgroundCallbacks(source="chat", location="options", injected={"temperature": 0.2}, raw=raw) + host = ( + InvocationsWorkflowHostServer( + callbacks.workflow, parse_input=lambda request: WorkflowTurn(input="hi", stream=stream) + ) + if workflow + else InvocationsHostServer(callbacks.agent) + ) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post( + "/invocations?agent_session_id=foreground", json={"message": "hi", "stream": stream} + ) + assert response.status_code == 200 + assert "ok" in response.text + assert callbacks.client.calls[0]["temperature"] == 0.2 + if not (workflow and raw): + assert callbacks.client.chat_middleware == [callbacks.inject_chat] diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_custom_storage_sample.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_custom_storage_sample.py new file mode 100644 index 00000000000..21336b04af1 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_custom_storage_sample.py @@ -0,0 +1,171 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Validate the optimistic-concurrency session store in the custom_storage hosting sample.""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path +from types import ModuleType +from typing import Any +from unittest.mock import MagicMock + +import pytest +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext +from azure.core import MatchConditions +from azure.cosmos.exceptions import CosmosHttpResponseError + +from .custom_storage_sample_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + FakeContainer, + cosmos_store, + local_store, + make_session, +) + +pytestmark = pytest.mark.filterwarnings("ignore:.*SessionStore is experimental.*") + +_SAMPLE = ( + Path(__file__).resolve().parents[4] + / "samples" + / "04-hosting" + / "foundry-hosted-agents" + / "responses" + / "custom_storage" + / "main.py" +) + + +@pytest.fixture(scope="module") +def sample() -> ModuleType: + spec = importlib.util.spec_from_file_location("custom_storage_sample", _SAMPLE) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +def _stores(sample: ModuleType, kind: str) -> tuple[Any, Any]: + if kind == "local": + snapshots: dict[Any, Any] = {} + return local_store(sample, snapshots), local_store(sample, snapshots) + container = FakeContainer() + return cosmos_store(sample, container), cosmos_store(sample, container) + + +async def test_sample_uses_only_the_public_session_store_contract() -> None: + source = _SAMPLE.read_text(encoding="utf-8") + assert "agent_framework_foundry_hosting._" not in source + assert "compare_and_set" not in source + assert "get_with_revision" not in source + + +@pytest.mark.parametrize("kind", ["local", "cosmos"]) +async def test_get_set_roundtrip_returns_independent_snapshots(sample: ModuleType, kind: str) -> None: + store, _ = _stores(sample, kind) + assert await store.get("alias") is None + original = make_session("one") + await store.set("alias", original) + original.state["value"] = "mutated" + loaded = await store.get("alias") + assert loaded is not None and loaded.state["value"] == "one" + loaded.state["value"] = "two" + await store.set("alias", loaded) + current = await store.get("alias") + assert current is not None and current.state["value"] == "two" + + +@pytest.mark.parametrize("kind", ["local", "cosmos"]) +async def test_stale_request_view_cannot_overwrite_newer_session(sample: ModuleType, kind: str) -> None: + first, second = _stores(sample, kind) + await first.set("alias", make_session("initial")) + assert await second.get("alias") is not None + await second.set("alias", make_session("newer")) + with pytest.raises(RuntimeError, match="reload before writing"): + await first.set("alias", make_session("stale")) + current = await second.get("alias") + assert current is not None and current.state["value"] == "newer" + + +@pytest.mark.parametrize("kind", ["local", "cosmos"]) +async def test_competing_creates_allow_only_one_writer(sample: ModuleType, kind: str) -> None: + first, second = _stores(sample, kind) + assert await first.get("alias") is None + assert await second.get("alias") is None + await first.set("alias", make_session("first")) + with pytest.raises(RuntimeError, match="reload before writing"): + await second.set("alias", make_session("second")) + current = await second.get("alias") + assert current is not None and current.state["value"] == "first" + + +@pytest.mark.parametrize("kind", ["local", "cosmos"]) +async def test_stale_delete_does_not_remove_another_requests_snapshot(sample: ModuleType, kind: str) -> None: + first, second = _stores(sample, kind) + await first.set("alias", make_session("initial")) + assert await second.get("alias") is not None + await second.set("alias", make_session("newer")) + with pytest.raises(RuntimeError): + await first.delete("alias") + current = await second.get("alias") + assert current is not None and current.state["value"] == "newer" + await second.delete("alias") + await second.delete("alias") + assert await second.get("alias") is None + + +async def test_local_store_scopes_are_isolated(sample: ModuleType) -> None: + snapshots: dict[Any, Any] = {} + first = local_store(sample, snapshots, "scope-a") + second = local_store(sample, snapshots, "scope-b") + await first.set("alias", make_session("a")) + assert await second.get("alias") is None + + +async def test_local_snapshots_are_shared_only_within_one_provider(sample: ModuleType) -> None: + config = MagicMock(spec=AgentConfig) + config.is_hosted = False + context = FoundryAgentRequestContext() + provider = sample.CustomSessionStoreProvider() + first = provider.get_store(config=config, platform_context=context) + same_provider = provider.get_store(config=config, platform_context=context) + other = sample.CustomSessionStoreProvider().get_store(config=config, platform_context=context) + + await first.set("alias", make_session("one")) + + assert first is not same_provider + stored = await same_provider.get("alias") + assert stored is not None and stored.state["value"] == "one" + assert await other.get("alias") is None + + +async def test_cosmos_uses_server_side_create_and_exact_etag_conditions(sample: ModuleType) -> None: + container = FakeContainer() + store = cosmos_store(sample, container) + await store.set("alias", make_session("one")) + await store.set("alias", make_session("two")) + await store.delete("alias") + assert container.calls == [ + ("create", {}), + ("replace", {"etag": '"1"', "match_condition": MatchConditions.IfNotModified}), + ("delete", {"etag": '"2"', "match_condition": MatchConditions.IfNotModified}), + ] + + +async def test_cosmos_rejects_stored_session_without_etag(sample: ModuleType) -> None: + container = FakeContainer() + store = cosmos_store(sample, container) + await store.set("alias", make_session("one")) + for item in container.items.values(): + del item["_etag"] + with pytest.raises(RuntimeError, match="missing its ETag"): + await cosmos_store(sample, container).get("alias") + + +async def test_cosmos_other_errors_propagate(sample: ModuleType) -> None: + container = FakeContainer() + container.create_error = 503 + with pytest.raises(CosmosHttpResponseError) as exc_info: + await cosmos_store(sample, container).set("alias", make_session("one")) + assert not isinstance(exc_info.value, RuntimeError) diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py new file mode 100644 index 00000000000..5d8013df22c --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py @@ -0,0 +1,358 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Verify supported settings and option boundaries without external model calls.""" + +from __future__ import annotations + +from typing import Any, cast +from unittest.mock import AsyncMock, MagicMock, patch + +import httpx +import pytest +from agent_framework import Agent, AgentContext, AgentSession, ChatOptions, RawAgent +from azure.ai.agentserver.responses import ResponsesServerOptions +from starlette.requests import Request + +from agent_framework_foundry_hosting import ( + InvocationsHostServer, + InvocationsWorkflowHostServer, + ResponsesRequest, + WorkflowTurn, +) +from agent_framework_foundry_hosting._scope import FoundryRequestScope +from agent_framework_foundry_hosting._shared import ( + _NativeAgentOptions, + build_agent_response_server_options, + build_workflow_response_server_options, + prepare_response_options, + response_run_options, + validate_agent_background_options, + validate_request_options, +) + +from .options_helpers import OptionCallbacks, OptionsClient + +_AGENT_DEFAULTS: dict[str, Any] = { + "history_source": "host_managed", + "background_source": "host_managed", + "max_history_items": None, + "sse_keep_alive_interval_seconds": None, + "shutdown_grace_period_seconds": 10, + "enable_background_recovery": False, +} +_WORKFLOW_DEFAULTS: dict[str, Any] = { + "sse_keep_alive_interval_seconds": None, + "shutdown_grace_period_seconds": 10, + "enable_background_recovery": False, +} + + +def _agent_options(**settings: Any) -> ResponsesServerOptions: + return build_agent_response_server_options(**{**_AGENT_DEFAULTS, **settings}) + + +def test_history_default_is_omitted_instead_of_forced_unlimited(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("DEFAULT_FETCH_HISTORY_ITEM_COUNT", "7") + with patch("agent_framework_foundry_hosting._shared.ResponsesServerOptions", wraps=ResponsesServerOptions) as sdk: + result = _agent_options() + assert "default_fetch_history_count" not in sdk.call_args.kwargs + assert result.default_fetch_history_count == ResponsesServerOptions().default_fetch_history_count + assert result.default_model is None + assert result.steerable_conversations is False + assert result.resilient_background is False + assert _agent_options(max_history_items=7).default_fetch_history_count == 7 + + +@pytest.mark.parametrize( + ("settings", "error"), + [ + ({"max_history_items": True}, TypeError), + ({"max_history_items": 1.5}, TypeError), + ({"max_history_items": 0}, ValueError), + ({"max_history_items": -1}, ValueError), + ({"sse_keep_alive_interval_seconds": True}, TypeError), + ({"sse_keep_alive_interval_seconds": "5"}, TypeError), + ({"sse_keep_alive_interval_seconds": 0}, ValueError), + ({"sse_keep_alive_interval_seconds": -1}, ValueError), + ({"shutdown_grace_period_seconds": True}, TypeError), + ({"shutdown_grace_period_seconds": 0}, ValueError), + ({"enable_background_recovery": 1}, TypeError), + ({"enable_background_recovery": True}, ValueError), + ({"background_source": "inference_service"}, ValueError), + ({"history_source": "wrong"}, ValueError), + ({"background_source": "wrong"}, ValueError), + ], +) +def test_invalid_agent_settings_fail_before_sdk_options_construction( + settings: dict[str, Any], error: type[Exception] +) -> None: + with patch("agent_framework_foundry_hosting._shared.ResponsesServerOptions") as sdk, pytest.raises(error): + _agent_options(**settings) + sdk.assert_not_called() + + +@pytest.mark.parametrize( + ("settings", "error"), + [ + ({"sse_keep_alive_interval_seconds": 0}, ValueError), + ({"shutdown_grace_period_seconds": True}, TypeError), + ({"enable_background_recovery": 1}, TypeError), + ], +) +def test_invalid_workflow_settings_fail_before_sdk_options_construction( + settings: dict[str, Any], error: type[Exception] +) -> None: + with patch("agent_framework_foundry_hosting._shared.ResponsesServerOptions") as sdk, pytest.raises(error): + build_workflow_response_server_options(**{**_WORKFLOW_DEFAULTS, **settings}) + sdk.assert_not_called() + + +def test_recovery_and_history_limit_eligibility() -> None: + assert _agent_options( + history_source="inference_service", + background_source="inference_service", + enable_background_recovery=True, + ).resilient_background + assert build_workflow_response_server_options(**{ + **_WORKFLOW_DEFAULTS, + "enable_background_recovery": True, + }).resilient_background + with pytest.warns(UserWarning, match="max_history_items is ignored"): + agent_managed_options = _agent_options(history_source="agent_managed", max_history_items=2) + assert agent_managed_options.default_fetch_history_count == ResponsesServerOptions().default_fetch_history_count + + +@pytest.mark.parametrize("env", [None, "0", "-2", "3"]) +@pytest.mark.parametrize("explicit", [None, 5]) +def test_invocations_heartbeat_precedence_and_per_host_isolation( + monkeypatch: pytest.MonkeyPatch, env: str | None, explicit: int | None +) -> None: + if env is None: + monkeypatch.delenv("SSE_KEEPALIVE_INTERVAL", raising=False) + else: + monkeypatch.setenv("SSE_KEEPALIVE_INTERVAL", env) + first = InvocationsHostServer(Agent(client=OptionsClient()), sse_keep_alive_interval_seconds=explicit) + second = InvocationsHostServer(Agent(client=OptionsClient())) + inherited = max(0, int(env or "0")) + assert first.config.sse_keepalive_interval == (explicit if explicit is not None else inherited) + assert second.config.sse_keepalive_interval == inherited + assert first.config is not second.config + + +@pytest.mark.parametrize("provider_background", [False, True]) +@pytest.mark.parametrize("background", [None, False, True, "true", 1]) +def test_agent_background_defaults_are_validated_without_mutation(provider_background: bool, background: Any) -> None: + agent = RawAgent(client=OptionsClient(), default_options=cast(ChatOptions, {"background": background})) + before = dict(agent.default_options) + if background is not None and not isinstance(background, bool): + with pytest.raises(TypeError, match="boolean"): + validate_agent_background_options(agent, provider_background=provider_background) + elif background is True and not provider_background: + with pytest.raises(ValueError, match="outer background"): + validate_agent_background_options(agent, provider_background=provider_background) + else: + validate_agent_background_options(agent, provider_background=provider_background) + assert agent.default_options == before + + +@pytest.mark.parametrize("provider_background", [False, True]) +def test_transport_body_cannot_bypass_background_selection(provider_background: bool) -> None: + agent = RawAgent(client=OptionsClient(), default_options=cast(ChatOptions, {"extra_body": {"background": True}})) + with pytest.raises(ValueError, match="extra_body"): + validate_agent_background_options(agent, provider_background=provider_background) + + +@pytest.mark.parametrize("field", ["additional_function_arguments", "client_kwargs", "function_invocation_kwargs"]) +def test_response_hook_cannot_supply_run_context(field: str) -> None: + with pytest.raises(ValueError, match=field): + validate_request_options({field: {"background": True}}) + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("async_hook", [False, True]) +async def test_invocations_hook_before_factory_preserves_developer_model(stream: bool, async_hook: bool) -> None: + callbacks = OptionCallbacks(defaults={"model": "developer", "temperature": 0.4}) + host = InvocationsHostServer( + callbacks.factory, + prepare_options=callbacks.prepare_async if async_hook else callbacks.prepare, + ) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post( + "/invocations?agent_session_id=options", + json={"message": "hi", "stream": stream, "options": {"model": "caller", "temperature": 0.8}}, + ) + assert response.status_code == 200 + assert callbacks.events == ["hook", "factory"] + assert callbacks.client.calls[0]["model"] == "developer" + assert callbacks.client.calls[0]["temperature"] == 0.8 + assert callbacks.defaults == {"model": "developer", "temperature": 0.4} + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_invocations_options_are_opt_in(stream: bool) -> None: + callbacks = OptionCallbacks(defaults={"model": "developer", "temperature": 0.4}) + host = InvocationsHostServer(callbacks.factory) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post( + "/invocations?agent_session_id=options", + json={"message": "hi", "stream": stream, "options": {"model": "caller", "temperature": 0.8}}, + ) + assert response.status_code == 200 + assert callbacks.client.calls[0]["model"] == "developer" + assert callbacks.client.calls[0]["temperature"] == 0.4 + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("background", [False, True]) +async def test_invocations_hook_background_is_passed_to_agent(stream: bool, background: bool) -> None: + callbacks = OptionCallbacks() + + def prepare(request: Request, options: dict[str, Any]) -> dict[str, Any]: + return {**options, "background": background} + + host = InvocationsHostServer(callbacks.factory, prepare_options=prepare) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/invocations?agent_session_id=options", json={"message": "hi", "stream": stream}) + assert response.status_code == 200 + assert callbacks.client.calls[0]["background"] is background + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_invocations_without_hook_passes_no_background(stream: bool) -> None: + callbacks = OptionCallbacks() + host = InvocationsHostServer(callbacks.factory) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post( + "/invocations?agent_session_id=options", + json={"message": "hi", "stream": stream, "options": {"background": True}}, + ) + assert response.status_code == 200 + assert "background" not in callbacks.client.calls[0] + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_invocations_rejects_factory_hidden_background_before_model_call( + stream: bool, caplog: pytest.LogCaptureFixture +) -> None: + callbacks = OptionCallbacks(defaults={"background": True}) + host = InvocationsHostServer(callbacks.factory, prepare_options=callbacks.prepare) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/invocations?agent_session_id=options", json={"message": "hi", "stream": stream}) + assert response.status_code == (200 if stream else 500) + assert "Agent invocation failed." in response.text + assert "background=True requires an outer background" in caplog.text + assert callbacks.events == ["hook", "factory"] + assert callbacks.client.calls == [] + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("source", ["defaults", "middleware"]) +async def test_native_workflow_rejects_inner_background_before_model_call( + stream: bool, source: str, caplog: pytest.LogCaptureFixture +) -> None: + callbacks = OptionCallbacks( + defaults={"background": True} if source == "defaults" else None, + injected={"background": True} if source == "middleware" else None, + ) + host = InvocationsWorkflowHostServer( + callbacks.workflow, parse_input=lambda request: WorkflowTurn(input="hi", stream=stream) + ) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/invocations?agent_session_id=workflow-options", json={}) + assert response.status_code == (200 if stream else 500 if source == "middleware" else 400) + if source == "middleware": + assert "Workflow invocation failed." in response.text + assert "Native workflow model options cannot override hosting" in caplog.text + else: + assert "hosting identity defaults" in response.text + assert callbacks.client.calls == [] + + +async def test_native_middleware_rechecks_direct_client_kwargs() -> None: + client = OptionsClient() + context = AgentContext( + agent=Agent(client=client), + messages=[], + session=AgentSession(), + client_kwargs={"background": True}, + ) + policy = _NativeAgentOptions(FoundryRequestScope("session", None, None, False)) + with pytest.raises(ValueError, match="client kwargs"): + await policy.process(context, cast(Any, None)) + assert client.calls == [] + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("injected", [{"background": True}, {"extra_body": {"background": True}}]) +async def test_native_middleware_rechecks_effective_options(stream: bool, injected: dict[str, Any]) -> None: + context = AgentContext( + agent=Agent(client=OptionsClient()), + messages=[], + session=AgentSession(), + stream=stream, + options=injected, + ) + policy = _NativeAgentOptions(FoundryRequestScope("session", None, None, False)) + dispatch = AsyncMock() + with pytest.raises(ValueError, match="hosting|host-controlled"): + await policy.process(context, dispatch) + dispatch.assert_not_called() + + +@pytest.mark.parametrize("workflow", [False, True]) +async def test_explicit_interval_emits_heartbeat_on_real_invocations_stream(workflow: bool) -> None: + callbacks = OptionCallbacks() + callbacks.client = OptionsClient(delay=1.1) + host = ( + InvocationsWorkflowHostServer( + callbacks.workflow, + parse_input=lambda request: WorkflowTurn(input="hi", stream=True), + sse_keep_alive_interval_seconds=1, + ) + if workflow + else InvocationsHostServer(callbacks.factory, sse_keep_alive_interval_seconds=1) + ) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/invocations?agent_session_id=heartbeat", json={"message": "hi", "stream": True}) + assert response.status_code == 200 + assert ": keep-alive\n\n" in response.text + assert "ok" in response.text + assert len(callbacks.client.calls) == 1 + + +async def test_response_hook_returns_independent_nested_options() -> None: + source: dict[str, Any] = {"metadata": {"source": "caller"}} + request = ResponsesRequest( + cast(Any, {}), + MagicMock(), + FoundryRequestScope("session", None, None, False), + source, + ) + await prepare_response_options(request, lambda request, options: options) + assert dict(request.options) == source + assert request.options["metadata"] is not source["metadata"] + with pytest.raises(TypeError, match="mapping"): + await prepare_response_options(request, lambda request, options: cast(Any, {1: "not-string"})) + + +async def test_response_options_are_opt_in_and_hook_can_select_known_fields() -> None: + caller_options = response_run_options( + cast(Any, {"extra_body": {"messages": "not-a-generation-option", "temperature": 0.8}}) + ) + request = ResponsesRequest( + cast(Any, {}), + MagicMock(), + FoundryRequestScope("session", None, None, False), + caller_options, + ) + await prepare_response_options(request, None) + assert dict(request.options) == {} + + selected_request = ResponsesRequest( + cast(Any, {}), + MagicMock(), + FoundryRequestScope("session", None, None, False), + caller_options, + ) + await prepare_response_options(selected_request, lambda _request, options: {"temperature": options["temperature"]}) + assert dict(selected_request.options) == {"temperature": 0.8} diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py index 81ef021147a..e0d5695fab6 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py @@ -55,12 +55,11 @@ from agent_framework_foundry_hosting import ( CheckpointStoreProvider, FoundryRequestScope, - InvocationsHostServer, InvocationsWorkflowHostServer, WorkflowSource, WorkflowTurn, ) -from agent_framework_foundry_hosting._invocations_shared import _workflow_output, _WorkflowOutputError +from agent_framework_foundry_hosting._invocations_workflow import _workflow_output, _WorkflowOutputError from agent_framework_foundry_hosting._state_store import FoundryCheckpointStore from agent_framework_foundry_hosting._workflow_state import FoundryWorkflowBindingStore, WorkflowHead @@ -438,20 +437,9 @@ def test_requires_one_source_and_an_explicit_parser(self) -> None: def create(_request: Request) -> Workflow: return WorkflowBuilder(name="tickets", start_executor=_TicketExecutor([])).build() - invalid_agent: Any = object() - with pytest.raises(TypeError, match="InvocationsWorkflowHostServer"): - InvocationsHostServer(agent=invalid_agent, workflow=create) with pytest.raises(TypeError, match="parse_input"): cast(Any, InvocationsWorkflowHostServer)(workflow=create) - def test_workflow_rejects_agent_legacy_wire_format(self) -> None: - with pytest.raises(TypeError, match="legacy_wire_format"): - InvocationsWorkflowHostServer( - workflow=lambda _: WorkflowBuilder(name="tickets", start_executor=_TicketExecutor([])).build(), - parse_input=_parse_ticket, - legacy_wire_format=True, - ) - @pytest.mark.parametrize("stream", [False, True]) async def test_parser_must_return_workflow_turn_before_building_or_loading_state(self, stream: bool) -> None: created: list[Request] = [] @@ -731,8 +719,8 @@ def factory(_request: Request) -> Workflow: async with raw_bindings: keys = await raw_bindings.list_keys(call_id=None) assert len(keys.keys) == 2 # Current response record plus the fixed lineage head. - # The current turn retains its entry and final checkpoints; prior turns are reclaimed. - assert len(await stores[-1].list_checkpoint_ids(workflow_name="tickets")) == 2 + # Reclaim runs after commit, so only the committed final checkpoint remains. + assert len(await stores[-1].list_checkpoint_ids(workflow_name="tickets")) == 1 before = list(calls) replay, events = await _invoke( @@ -796,7 +784,7 @@ async def handle(self, _: Ticket, ctx: WorkflowContext[None, Any]) -> None: class TestNativeWorkflowPendingReplies: @pytest.mark.parametrize("stream", [False, True]) - async def test_json_approval_metadata_is_bound_to_exact_pending_calls_before_dispatch(self, stream: bool) -> None: + async def test_json_approval_replies_dispatch_once_and_replay_is_rejected(self, stream: bool) -> None: decisions: list[bool] = [] async def parser(request: Request) -> WorkflowTurn[Ticket]: @@ -818,18 +806,6 @@ def factory(_request: Request) -> Workflow: item["request_id"]: Content.from_dict(item["data"]).to_function_approval_response(True).to_dict() for item in pending } - request_ids = list(valid) - duplicated = {request_ids[0]: valid[request_ids[0]], request_ids[1]: valid[request_ids[0]]} - response, rejected = await _invoke(_host(factory, parser=parser), {"responses": duplicated, "stream": stream}) - assert response.status_code == (200 if stream else 400) - assert [event["event"] for event in rejected] == ["error"] - assert decisions == [] - forged = json.loads(json.dumps(valid)) - forged[request_ids[0]]["function_call"]["arguments"] = {"ticket_id": "forged"} - response, rejected = await _invoke(_host(factory, parser=parser), {"responses": forged, "stream": stream}) - assert response.status_code == (200 if stream else 400) - assert [event["event"] for event in rejected] == ["error"] - assert decisions == [] response, _ = await _invoke(_host(factory, parser=parser), {"responses": valid, "stream": stream}) assert response.status_code == 200 assert decisions == [True, True] @@ -887,9 +863,7 @@ def factory(_request: Request) -> Workflow: assert replies == [{"accepted": True}, {"accepted": False}] @pytest.mark.parametrize("stream", [False, True]) - async def test_partial_replies_do_not_consume_authority_and_complete_replies_survive_restart( - self, stream: bool - ) -> None: + async def test_partial_replies_dispatch_answered_requests_and_keep_the_rest_pending(self, stream: bool) -> None: replies: list[tuple[str, bool]] = [] def factory(_request: Request) -> Workflow: @@ -900,28 +874,25 @@ def factory(_request: Request) -> Workflow: pending = [item for item in events if item["event"] == "request_info"] assert len(pending) == 2 ids = {item["data"]["reviewer"]: item["request_id"] for item in pending} - first_reply = {"responses": {ids["support"]: {"approved": True}}, "stream": stream} - response, events = await _invoke(_host(factory), first_reply) - assert response.status_code == (200 if stream else 400) - assert [item["event"] for item in events] == ["error"] - assert replies == [] + response, _ = await _invoke( + _host(factory), {"responses": {ids["support"]: {"approved": True}}, "stream": stream} + ) + assert response.status_code == 200 + assert replies == [("support", True)] - complete = { - "responses": {ids["support"]: {"approved": True}, ids["security"]: {"approved": False}}, - "stream": stream, - } - response, events = await _invoke(_host(factory), complete) + rest = {"responses": {ids["security"]: {"approved": False}}, "stream": stream} + response, events = await _invoke(_host(factory), rest) assert response.status_code == 200 - assert sorted(replies) == [("security", False), ("support", True)] + assert replies == [("support", True), ("security", False)] assert not [item for item in events if item["event"] == "request_info"] - response, events = await _invoke(_host(factory), complete) + response, events = await _invoke(_host(factory), rest) assert response.status_code == (200 if stream else 400) assert [item["event"] for item in events] == ["error"] assert len(replies) == 2 - @pytest.mark.parametrize("identity", ["other-sandbox", "other-user", "forged-request"]) - async def test_pending_reply_cannot_cross_scope_or_forge_a_request(self, identity: str) -> None: + @pytest.mark.parametrize("identity", ["other-sandbox", "other-user"]) + async def test_pending_reply_cannot_cross_scope(self, identity: str) -> None: replies: list[tuple[str, bool]] = [] def factory(_request: Request) -> Workflow: @@ -940,7 +911,7 @@ def factory(_request: Request) -> Workflow: other.config.session_id = "" response, _ = await _invoke( other, - {"responses": {"forged" if identity == "forged-request" else request_id: {"approved": True}}}, + {"responses": {request_id: {"approved": True}}}, session_id="sandbox-b" if identity == "other-sandbox" else "sandbox-a", user_id="user-b" if identity == "other-user" else "user-a", call_id="second", @@ -965,7 +936,7 @@ def factory(_request: Request) -> Workflow: assert response.status_code == 200 assert len(replies) == 2 - async def test_invalid_mixed_reply_batch_does_not_consume_valid_authority(self) -> None: + async def test_unknown_reply_id_fails_in_core_without_dispatch_and_valid_retry_succeeds(self) -> None: replies: list[tuple[str, bool]] = [] def factory(_request: Request) -> Workflow: @@ -976,7 +947,8 @@ def factory(_request: Request) -> Workflow: response, _ = await _invoke( _host(factory), {"responses": {ids[0]: {"approved": True}, "forged": {"approved": True}}} ) - assert response.status_code == 400 + # Core rejects the unknown request ID before the first superstep; the claim is released, not blocked. + assert response.status_code == 500 assert replies == [] response, _ = await _invoke(_host(factory), {"responses": {key: {"approved": True} for key in ids}}) assert response.status_code == 200 @@ -1273,11 +1245,11 @@ async def handle(self, _: Ticket, ctx: WorkflowContext[None, str]) -> None: byte_limit = 4096 if limit_kind == "events" else 128 with ( patch( - "agent_framework_foundry_hosting._invocations_shared._MAX_WORKFLOW_SNAPSHOT_EVENTS", + "agent_framework_foundry_hosting._invocations_workflow._MAX_WORKFLOW_SNAPSHOT_EVENTS", event_limit, ), patch( - "agent_framework_foundry_hosting._invocations_shared._MAX_WORKFLOW_SNAPSHOT_BYTES", + "agent_framework_foundry_hosting._invocations_workflow._MAX_WORKFLOW_SNAPSHOT_BYTES", byte_limit, ), ): diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py index 14d9b1f11d3..7e946b99266 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py @@ -7,7 +7,7 @@ import logging from collections.abc import AsyncIterator, Awaitable, Callable, Mapping, MutableMapping, Sequence from typing import Any, cast -from unittest.mock import patch +from unittest.mock import AsyncMock, MagicMock, patch import httpx import pytest @@ -35,8 +35,15 @@ ) from agent_framework._workflows._agent_utils import prepare_executor_run_kwargs from agent_framework._workflows._const import RESOLVED_WORKFLOW_RUN_KWARGS_KEY, WORKFLOW_RUN_KWARGS_KEY -from azure.ai.agentserver.responses import FileResponseStore, InMemoryResponseProvider, ResponsesServerOptions +from azure.ai.agentserver.responses import FileResponseStore, InMemoryResponseProvider from azure.ai.agentserver.responses._id_generator import IdGenerator +from azure.ai.agentserver.responses.models import ( + CreateResponse, + ItemReferenceParam, + MCPApprovalResponse, + OutputItem, + ResponseObject, +) from pydantic import BaseModel from agent_framework_foundry_hosting import ( @@ -47,6 +54,7 @@ ResponsesWorkflowHostServer, WorkflowTurn, ) +from agent_framework_foundry_hosting._responses_workflow import _load_replies, _WorkflowRequestError from agent_framework_foundry_hosting._workflow_state import FoundryWorkflowBindingStore, HostedWorkflowRun @@ -380,7 +388,7 @@ async def test_graph_change_cannot_choose_a_new_unrelated_checkpoint() -> None: assert "graph" in second["error"]["message"] -async def test_generic_pending_replies_are_complete_and_consumed_once() -> None: +async def test_generic_partial_replies_reproject_unanswered_requests_under_new_ids() -> None: server = _server(lambda request: _build(_Review()), parser=_parse_text) first = await _post(server, "review") assert first["status"] == "completed", first @@ -391,19 +399,27 @@ async def test_generic_pending_replies_are_complete_and_consumed_once() -> None: [{"type": "function_call_output", "call_id": call_ids[0], "output": "true"}], previous_response_id=first["id"], ) - assert partial["status"] == "failed" - assert not partial["output"] - full = [ - {"type": "function_call_output", "call_id": call_ids[0], "output": "true"}, - {"type": "function_call_output", "call_id": call_ids[1], "output": "false"}, - ] - duplicate = await _post(server, [full[0], dict(full[0]), full[1]], previous_response_id=first["id"]) + assert partial["status"] == "completed", partial + assert _texts(partial) == ["review:True"] + new_ids = [item["call_id"] for item in partial["output"] if item["type"] == "function_call"] + assert len(new_ids) == 1 and new_ids[0] not in call_ids + reply = {"type": "function_call_output", "call_id": new_ids[0], "output": "false"} + + duplicate = await _post(server, [reply, dict(reply)], previous_response_id=partial["id"]) assert duplicate["status"] == "failed" assert not duplicate["output"] - completed = await _post(server, full, previous_response_id=first["id"]) + stale = await _post( + server, + [{"type": "function_call_output", "call_id": call_ids[1], "output": "false"}], + previous_response_id=partial["id"], + ) + assert stale["status"] == "failed" + assert not stale["output"] + + completed = await _post(server, [reply], previous_response_id=partial["id"]) assert completed["status"] == "completed", completed - assert _texts(completed) == ["review:True", "review:False"] - replay = await _post(server, full, previous_response_id=first["id"]) + assert _texts(completed) == ["review:False"] + replay = await _post(server, [reply], previous_response_id=partial["id"]) assert replay["status"] == "failed" assert not replay["output"] @@ -428,31 +444,257 @@ async def test_named_conversation_rejects_old_wire_replies_when_core_request_ids assert _texts(accepted) == ["new question:False", "new question:False"] -async def test_approval_wire_ids_bind_only_to_exact_pending_checkpoint() -> None: +@pytest.mark.parametrize("stream", [False, True]) +async def test_workflow_binding_lookup_error_fails_without_leaking_detail(stream: bool) -> None: server = _server(lambda request: _build(_Approvals()), parser=_parse_text) first = await _post(server, "/safe") + approval_id = next(item["id"] for item in first["output"] if item["type"] == "mcp_approval_request") + detail = "private workflow storage failure" + + with patch.object(FoundryWorkflowBindingStore, "get_response", new=AsyncMock(side_effect=RuntimeError(detail))): + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + response = await client.post( + "/responses", + headers={"x-agent-user-id": "user", "x-agent-foundry-call-id": "call"}, + json={ + "input": [{"type": "mcp_approval_response", "approval_request_id": approval_id, "approve": True}], + "agent_session_id": "sandbox", + "previous_response_id": first["id"], + "stream": stream, + }, + ) + + assert response.status_code == 200 + assert detail not in response.text + if stream: + assert "response.failed" in response.text + else: + assert response.json()["status"] == "failed" + + +def _terminal(response: httpx.Response, stream: bool) -> dict[str, Any]: + assert response.status_code == 200, response.text + if not stream: + return response.json() + events = [json.loads(line[6:]) for line in response.text.splitlines() if line.startswith("data: {")] + return next( + event["response"] + for event in events + if event["type"] in ("response.completed", "response.failed", "response.incomplete") + ) + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_invalid_approval_replies_fail_in_stream_and_leave_the_pause_usable(stream: bool) -> None: + provider = InMemoryResponseProvider() + server = _server(lambda request: _build(_Approvals()), parser=_parse_text, response_store=provider) + first = await _post(server, "/safe") assert first["status"] == "completed", first wire_ids = [item["id"] for item in first["output"] if item["type"] == "mcp_approval_request"] assert len(wire_ids) == 2 full = [ - {"type": "mcp_approval_response", "approval_request_id": wire_ids[0], "approve": True}, - {"type": "mcp_approval_response", "approval_request_id": wire_ids[1], "approve": False}, + MCPApprovalResponse(type="mcp_approval_response", approval_request_id=wire_ids[0], approve=True), + MCPApprovalResponse(type="mcp_approval_response", approval_request_id=wire_ids[1], approve=False), ] - for replies in ( - full[:1], - [full[0], dict(full[0]), full[1]], - [{**full[0], "approval_request_id": "forged"}, full[1]], - ): - rejected = await _post(server, replies, previous_response_id=first["id"]) - assert rejected["status"] == "failed" - assert not rejected["output"] - wrong_scope = await _post(server, full, sandbox="other", previous_response_id=first["id"]) - assert wrong_scope["status"] == "failed" - completed = await _post(server, full, previous_response_id=first["id"]) + headers = {"x-agent-user-id": "user", "x-agent-foundry-call-id": "call"} + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + for replies, sandbox in ( + ([full[0], dict(full[0]), full[1]], "sandbox"), + ([*full, {"type": "item_reference", "id": "missing-item"}], "sandbox"), + (full, "other"), + ): + rejected = _terminal( + await client.post( + "/responses", + headers=headers, + json={ + "input": replies, + "agent_session_id": sandbox, + "previous_response_id": first["id"], + "stream": stream, + }, + ), + stream, + ) + assert rejected["status"] == "failed" + assert not rejected["output"] + redelivery = _terminal( + await client.post( + "/responses", + headers={**headers, "x-agent-response-id": first["id"]}, + json={ + "input": full, + "agent_session_id": "sandbox", + "previous_response_id": first["id"], + "stream": stream, + }, + ), + stream, + ) + assert redelivery["status"] == "failed" + assert not redelivery["output"] + direct_request = ResponsesRequest( + CreateResponse(input=[*full, ItemReferenceParam(type="item_reference", id="missing-item")]), + MagicMock(response_id="handler-revalidation", get_input_items=AsyncMock(return_value=full)), + MagicMock(spec=FoundryRequestScope), + {}, + ) + pending_run = MagicMock(spec=HostedWorkflowRun) + pending_run.pending_requests = {"first": object(), "second": object()} + with pytest.raises(_WorkflowRequestError, match="unresolved input references"): + await _load_replies(direct_request, pending_run) + + referenced_id = "resolved-decision" + await provider.create_response( + cast(ResponseObject, {**first, "id": "reference-seed", "output": []}), + [cast(OutputItem, {**full[0], "id": referenced_id})], + [], + ) + continuation_id = IdGenerator.new_response_id(first["id"]) + continuation = [{"type": "item_reference", "id": referenced_id}, full[1]] + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + completed = _terminal( + await client.post( + "/responses", + headers=headers, + json={ + "input": continuation, + "agent_session_id": "sandbox", + "previous_response_id": first["id"], + "response_id": continuation_id, + "stream": stream, + }, + ), + stream, + ) + assert completed["status"] == "completed", completed + assert completed["id"] == continuation_id + assert _texts(completed) == ["first:True", "second:False"] + replay = _terminal( + await client.post( + "/responses", + headers=headers, + json={ + "input": full, + "agent_session_id": "sandbox", + "previous_response_id": first["id"], + "stream": stream, + }, + ), + stream, + ) + assert replay["status"] == "failed" + assert not replay["output"] + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_rejected_typed_reply_fails_then_same_parent_accepts_a_valid_retry(stream: bool) -> None: + server = _server(lambda request: _build(_Review()), parser=_parse_text) + first = await _post(server, "review") + call_ids = [item["call_id"] for item in first["output"]] + headers = {"x-agent-user-id": "user", "x-agent-foundry-call-id": "call"} + + async def reply(output: str) -> dict[str, Any]: + replies = [{"type": "function_call_output", "call_id": call_id, "output": output} for call_id in call_ids] + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + response = await client.post( + "/responses", + headers=headers, + json={ + "input": replies, + "agent_session_id": "sandbox", + "previous_response_id": first["id"], + "stream": stream, + }, + ) + return _terminal(response, stream) + + rejected = await reply('{"not": "a bool"}') + assert rejected["status"] == "failed" + assert not rejected["output"] + completed = await reply("true") + assert completed["status"] == "completed", completed + assert _texts(completed) == ["review:True", "review:True"] + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_forged_and_partial_approvals_execute_only_pending_answers(stream: bool) -> None: + server = _server(lambda request: _build(_Approvals()), parser=_parse_text) + first = await _post(server, "/safe") + wire_ids = [item["id"] for item in first["output"] if item["type"] == "mcp_approval_request"] + headers = {"x-agent-user-id": "user", "x-agent-foundry-call-id": "call"} + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + + async def reply(previous_id: str, replies: list[dict[str, Any]]) -> dict[str, Any]: + return _terminal( + await client.post( + "/responses", + headers=headers, + json={ + "input": replies, + "agent_session_id": "sandbox", + "previous_response_id": previous_id, + "stream": stream, + }, + ), + stream, + ) + + partial = await reply( + first["id"], + [ + {"type": "mcp_approval_response", "approval_request_id": "forged", "approve": True}, + {"type": "mcp_approval_response", "approval_request_id": wire_ids[1], "approve": False}, + ], + ) + assert partial["status"] == "completed", partial + assert _texts(partial) == ["second:False"] + remaining = [item["id"] for item in partial["output"] if item["type"] == "mcp_approval_request"] + assert len(remaining) == 1 and remaining[0] not in wire_ids + + stale = await reply( + partial["id"], [{"type": "mcp_approval_response", "approval_request_id": wire_ids[0], "approve": True}] + ) + assert stale["status"] == "failed" + assert not stale["output"] + completed = await reply( + partial["id"], [{"type": "mcp_approval_response", "approval_request_id": remaining[0], "approve": True}] + ) + assert completed["status"] == "completed", completed + assert _texts(completed) == ["first:True"] + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_named_conversation_rejects_new_input_while_approvals_are_pending(stream: bool) -> None: + server = _server(lambda request: _build(_Approvals()), parser=_parse_text) + conversation = IdGenerator.new_id("conv") + first = await _post(server, "/safe", conversation=conversation) + wire_ids = [item["id"] for item in first["output"] if item["type"] == "mcp_approval_request"] + assert len(wire_ids) == 2 + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + for id_in_header in (None, True, False): + headers = {"x-agent-user-id": "user", "x-agent-foundry-call-id": "call"} + payload = { + "input": "a new request", + "agent_session_id": "sandbox", + "conversation": conversation, + "stream": stream, + } + if id_in_header is True: + headers["x-agent-response-id"] = first["id"] + elif id_in_header is False: + payload["response_id"] = first["id"] + rejected = _terminal(await client.post("/responses", headers=headers, json=payload), stream) + assert rejected["status"] == "failed" + assert not rejected["output"] + + replies = [ + {"type": "mcp_approval_response", "approval_request_id": wire_id, "approve": index == 0} + for index, wire_id in enumerate(wire_ids) + ] + completed = await _post(server, replies, conversation=conversation) assert completed["status"] == "completed", completed assert _texts(completed) == ["first:True", "second:False"] - replay = await _post(server, full, previous_response_id=first["id"]) - assert replay["status"] == "failed" async def test_store_false_has_no_durable_native_state_and_cannot_pause() -> None: @@ -486,7 +728,7 @@ async def test_unsupported_shared_workflow_is_not_cloned_or_unwrapped() -> None: ResponsesWorkflowHostServer( workflow=cast(Any, _build()), parse_input=_parse, - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, ) @@ -510,10 +752,21 @@ def create(request: ResponsesRequest) -> Workflow: @pytest.mark.parametrize("stream", [False, True]) -async def test_workflow_host_prefix_preserves_create_and_retrieval_routes(stream: bool) -> None: - server = _server(prefix="api/v1") +@pytest.mark.parametrize( + ("prefix", "path"), + [ + ("api/v1", "/api/v1"), + (" /api/v1/ ", "/api/v1"), + ("//api/v1/", "//api/v1"), + (" / ", ""), + ], +) +async def test_workflow_host_prefix_preserves_create_and_retrieval_routes(stream: bool, prefix: str, path: str) -> None: + server = _server(prefix=prefix) async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: - created = await client.post("/api/v1/responses", json={"input": '{"text":"prefixed"}', "stream": stream}) + created = await client.post( + f"http://test{path}/responses", json={"input": '{"text":"prefixed"}', "stream": stream} + ) assert created.status_code == 200 if stream: events = [json.loads(line[6:]) for line in created.text.splitlines() if line.startswith("data: {")] @@ -522,7 +775,7 @@ async def test_workflow_host_prefix_preserves_create_and_retrieval_routes(stream result = created.json() assert result["status"] == "completed", result assert result["output"][0]["output"] == {"text": "prefixed", "count": 1} - stored = await client.get(f"/api/v1/responses/{result['id']}") + stored = await client.get(f"http://test{path}/responses/{result['id']}") assert stored.status_code == 200 assert stored.json()["output"] == result["output"] @@ -645,7 +898,10 @@ async def start(self, message: str, ctx: WorkflowContext[str, Any]) -> None: assert message["content"][0]["annotations"] == [_expected_url_citation(start_index=4, end_index=10)] -async def test_native_agent_executor_applies_options_without_mutating_defaults_or_double_keywords() -> None: +@pytest.mark.parametrize("forward_options", [False, True]) +async def test_native_agent_executor_applies_options_without_mutating_defaults_or_double_keywords( + forward_options: bool, +) -> None: calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] agents: list[Agent[Any]] = [] @@ -660,7 +916,10 @@ def create(request: ResponsesRequest) -> Workflow: return _build(AgentExecutor(agent, id="model")) async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: - return WorkflowTurn(input=[Message("user", await request.get_input_text() or "")]) + return WorkflowTurn( + input=[Message("user", await request.get_input_text() or "")], + client_kwargs={"options": dict(request.options)} if forward_options else None, + ) server = _server(create, parser=parse) first = await _post(server, "one", temperature=0.7, max_output_tokens=12, store=True) @@ -668,8 +927,11 @@ async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: second = await _post(server, "two", temperature=0.4, store=False, call="call-two") assert second["status"] == "completed", second assert [call["options"]["store"] for call in calls] == [False, False] - assert [call["options"]["temperature"] for call in calls] == [0.7, 0.4] - assert calls[0]["options"]["max_tokens"] == 12 + assert [call["options"]["temperature"] for call in calls] == ([0.7, 0.4] if forward_options else [0.2, 0.2]) + if forward_options: + assert calls[0]["options"]["max_tokens"] == 12 + else: + assert all("max_tokens" not in call["options"] for call in calls) assert all( agent.default_options["temperature"] == 0.2 and agent.default_options["store"] is True for agent in agents ) @@ -690,7 +952,14 @@ def create(request: ResponsesRequest) -> Workflow: ) return _build(_RegistryExecutor(agent)) - response = await _post(_server(create, parser=_parse_text), "hello", temperature=0.7) + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: + return WorkflowTurn(input=await request.get_input_text(), client_kwargs={"options": dict(request.options)}) + + response = await _post( + _server(create, parser=parse), + "hello", + temperature=0.7, + ) assert response["status"] == "completed", response assert _texts(response) == ["safe"] @@ -767,7 +1036,10 @@ def create(request: ResponsesRequest) -> Workflow: return _build(AgentExecutor(agent, id="raw")) async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: - return WorkflowTurn(input=[Message("user", await request.get_input_text() or "")]) + return WorkflowTurn( + input=[Message("user", await request.get_input_text() or "")], + client_kwargs={"options": dict(request.options)}, + ) server = _server(create, parser=parse) async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: @@ -903,6 +1175,32 @@ async def test_real_sdk_background_cancel_drains_suspended_executor_and_blocks_c assert record is not None and record.status == "blocked" +@pytest.mark.parametrize( + ("prefix", "path"), [("api/v1", "/api/v1"), (" /api/v1/ ", "/api/v1"), ("//api/v1/", "//api/v1")] +) +async def test_prefixed_workflow_background_cancel_uses_sdk_routes(prefix: str, path: str) -> None: + started, release, stopped = (asyncio.Event() for _ in range(3)) + server = _server(_progress_factory(started, release, stopped), parser=_parse_count, prefix=prefix) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + created = await client.post( + f"http://test{path}/responses", + json={"input": "2", "agent_session_id": "sandbox", "store": True, "background": True}, + ) + assert created.status_code == 200, created.text + response_id = created.json()["id"] + await asyncio.wait_for(started.wait(), 2) + cancelled = await client.post(f"http://test{path}/responses/{response_id}/cancel") + assert cancelled.status_code == 200, cancelled.text + await asyncio.wait_for(stopped.wait(), 2) + for _ in range(150): + fetched = await client.get(f"http://test{path}/responses/{response_id}") + if fetched.status_code == 200 and fetched.json()["status"] == "cancelled": + break + await asyncio.sleep(0.01) + assert fetched.status_code == 200, fetched.text + assert fetched.json()["status"] == "cancelled" + + async def test_real_sdk_raw_agent_cancel_closes_request_owned_client_in_trusted_context() -> None: from azure.ai.agentserver.core import get_request_context @@ -1052,8 +1350,13 @@ async def update_response(self, response: Any, **kwargs: Any) -> None: provider = FailingCheckpointStore(storage_dir=tmp_path / "responses") factory = _progress_factory(started, release, stopped) - options = ResponsesServerOptions(resilient_background=True, shutdown_grace_period_seconds=1) - server = _server(factory, parser=_parse_count, response_store=provider, options=options) + server = _server( + factory, + parser=_parse_count, + response_store=provider, + enable_background_recovery=True, + shutdown_grace_period_seconds=1, + ) paired = asyncio.Event() original = HostedWorkflowRun.stage @@ -1090,7 +1393,8 @@ async def stage(run: HostedWorkflowRun, snapshot: Any, **kwargs: Any) -> None: factory, parser=_parse_count, response_store=provider, - options=ResponsesServerOptions(resilient_background=True, shutdown_grace_period_seconds=1), + enable_background_recovery=True, + shutdown_grace_period_seconds=1, ) async with ( restarted.router.lifespan_context(restarted), diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_oauth_consent_invocation.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_oauth_consent_invocation.py new file mode 100644 index 00000000000..9ec974446ed --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_oauth_consent_invocation.py @@ -0,0 +1,342 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Invocation-time Foundry Toolbox OAuth consent and its Responses translation.""" + +from __future__ import annotations + +import asyncio +import contextvars +from collections.abc import Mapping +from typing import Any + +import pytest +from agent_framework import Agent, ChatOptions, Content, Message, UserInputRequiredException +from agent_framework.exceptions import ToolExecutionException +from azure.ai.agentserver.responses.aio import ResponseEventStream +from mcp import MCPError + +from agent_framework_foundry_hosting._responses_shared import ( + OAUTH_CONSENT_NOT_EXECUTED_RESULT, + OAUTH_CONSENT_REQUIRED_RESULT, + _activate_toolbox_consent_scope, + _current_toolbox_consent_scope, + _normalize_allowed_oauth_consent_origins, + _OutputItemTracker, + tool_call_consent_url_from_error, +) + +from .toolbox_consent_helpers import ( + CONSENT_LINK, + BatchToolCallClient, + ToolboxConsentServer, + consent_toolbox, +) + +# region Error parsing + + +@pytest.mark.parametrize( + ("error", "expected"), + [ + (MCPError(-32006, "Request failed (remote): https://c.example.com/x"), "https://c.example.com/x"), + (MCPError(-32006, "ignored", {"detail": " https://d.example.com/y "}), "https://d.example.com/y"), + (MCPError(-32006, "Request failed (remote): https://c.example.com/x", {"detail": 7}), ""), + ( + MCPError( + -32006, + 'tools/list failed {"errors": [{"name": "s", "type": "mcp", "error": ' + '{"code": "CONSENT_REQUIRED", "message": "https://j.example.com/z"}}]}', + ), + "https://j.example.com/z", + ), + (MCPError(-32000, "Request failed (remote): https://c.example.com/x"), None), + ], +) +def test_tool_call_consent_url_from_error_parses_gateway_shapes(error: MCPError, expected: str | None) -> None: + assert tool_call_consent_url_from_error(error) == expected + assert tool_call_consent_url_from_error(ToolExecutionException("call failed", error)) == expected + + +def test_tool_call_consent_url_from_error_ignores_unrelated_exceptions() -> None: + assert tool_call_consent_url_from_error(ToolExecutionException("boom")) is None + assert tool_call_consent_url_from_error(ValueError("https://c.example.com")) is None + + +# endregion + +# region Scope + + +async def test_consent_scope_is_shared_with_copied_task_contexts() -> None: + async def run() -> None: + scope = _activate_toolbox_consent_scope() + + def mark() -> None: + current = _current_toolbox_consent_scope() + assert current is not None + assert current is scope + current.consent_rejected = True + + contextvars.copy_context().run(mark) + await asyncio.create_task(asyncio.sleep(0)) + assert scope.blocks_toolbox_calls + + await asyncio.create_task(run()) + assert _current_toolbox_consent_scope() is None + + +# endregion + +# region Toolbox call_tool + + +async def _in_scope(coroutine_factory: Any, *, allowed_origins: frozenset[str] | None = None) -> Any: + async def run() -> Any: + scope = _activate_toolbox_consent_scope(allowed_origins=allowed_origins) + try: + return scope, await coroutine_factory() + except BaseException as exc: # noqa: BLE001 - returned for assertions + return scope, exc + + return await asyncio.create_task(run()) + + +async def test_call_tool_translates_allowed_consent_to_structured_user_input() -> None: + server = ToolboxConsentServer(["secure"]) + server.require_consent("secure") + async with consent_toolbox(server) as toolbox: + scope, result = await _in_scope(lambda: toolbox.call_tool("secure")) + + assert isinstance(result, UserInputRequiredException) + function_result, consent = result.contents + assert function_result.type == "function_result" + assert function_result.result == OAUTH_CONSENT_REQUIRED_RESULT + assert consent.type == "oauth_consent_request" + assert consent.consent_link == CONSENT_LINK + assert consent.user_input_request is True + assert scope.consent_required and scope.blocks_toolbox_calls + assert [(c.name, c.consent_url) for c in scope.consents] == [(toolbox.name, CONSENT_LINK)] + + +async def test_call_tool_uses_structured_detail_for_consent_link() -> None: + server = ToolboxConsentServer(["secure"]) + server.require_consent("secure", message="consent required", data={"detail": CONSENT_LINK}) + async with consent_toolbox(server) as toolbox: + _, result = await _in_scope(lambda: toolbox.call_tool("secure")) + + assert isinstance(result, UserInputRequiredException) + assert result.contents[1].consent_link == CONSENT_LINK + + +@pytest.mark.parametrize( + ("message", "allowed"), + [ + ("Request failed (remote): http://consent.example.com/authorize", None), + ("Request failed (remote): https://user:pw@consent.example.com/authorize", None), + (f"Request failed (remote): {CONSENT_LINK}", ["https://other.example.com"]), + ("Request failed (remote): ", None), + ], +) +async def test_call_tool_rejects_unsafe_or_disallowed_consent_links(message: str, allowed: list[str] | None) -> None: + server = ToolboxConsentServer(["secure"]) + server.require_consent("secure", message=message) + async with consent_toolbox(server) as toolbox: + scope, result = await _in_scope( + lambda: toolbox.call_tool("secure"), + allowed_origins=_normalize_allowed_oauth_consent_origins(allowed), + ) + + assert isinstance(result, ToolExecutionException) + assert not isinstance(result, UserInputRequiredException) + assert "consent.example.com" not in str(result) + assert result.__cause__ is None + assert scope.consent_rejected and not scope.consent_required and scope.blocks_toolbox_calls + + +async def test_call_tool_accepts_consent_link_from_allowed_origin() -> None: + server = ToolboxConsentServer(["secure"]) + server.require_consent("secure") + async with consent_toolbox(server) as toolbox: + _, result = await _in_scope( + lambda: toolbox.call_tool("secure"), + allowed_origins=_normalize_allowed_oauth_consent_origins(["https://consent.example.com"]), + ) + + assert isinstance(result, UserInputRequiredException) + + +async def test_call_tool_blocks_later_toolbox_calls_without_contacting_gateway() -> None: + server = ToolboxConsentServer(["secure", "open"]) + server.require_consent("secure") + async with consent_toolbox(server) as toolbox: + + async def both() -> Any: + with pytest.raises(UserInputRequiredException): + await toolbox.call_tool("secure") + return await toolbox.call_tool("open") + + _, result = await _in_scope(both) + + assert result == OAUTH_CONSENT_NOT_EXECUTED_RESULT + assert server.calls == ["secure"] + + +async def test_call_tool_without_scope_propagates_original_error() -> None: + server = ToolboxConsentServer(["secure"]) + server.require_consent("secure") + async with consent_toolbox(server) as toolbox: + with pytest.raises(ToolExecutionException) as error: + await toolbox.call_tool("secure") + + assert not isinstance(error.value, UserInputRequiredException) + assert tool_call_consent_url_from_error(error.value) == CONSENT_LINK + + +async def test_call_tool_in_scope_propagates_non_consent_errors() -> None: + server = ToolboxConsentServer(["secure"]) + server.consent_errors["secure"] = {"code": -32000, "message": "boom"} + async with consent_toolbox(server) as toolbox: + scope, result = await _in_scope(lambda: toolbox.call_tool("secure")) + + assert isinstance(result, ToolExecutionException) + assert not scope.blocks_toolbox_calls + + +# endregion + +# region Function loop + + +def _function_results(messages: list[Message]) -> dict[str, list[Content]]: + results: dict[str, list[Content]] = {} + for message in messages: + for content in message.contents: + if content.type == "function_result" and content.call_id is not None: + results.setdefault(content.call_id, []).append(content) + return results + + +def _all_contents(messages: list[Message]) -> list[Content]: + return [content for message in messages for content in message.contents] + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_sequential_function_loop_returns_consent_and_blocks_later_calls(stream: bool) -> None: + server = ToolboxConsentServer(["before", "secure", "after"]) + server.require_consent("secure") + async with consent_toolbox(server) as toolbox: + client = BatchToolCallClient([function.name for function in toolbox.functions]) + agent = Agent(client=client, tools=toolbox, default_options=ChatOptions(allow_multiple_tool_calls=True)) + + async def run() -> list[Message]: + client.function_invocation_configuration["allow_concurrent_invocation"] = False # type: ignore[index] + if stream: + response_stream = agent.run("go", stream=True) + async for _ in response_stream: + pass + return list((await response_stream.get_final_response()).messages) + return list((await agent.run("go")).messages) + + scope, messages = await _in_scope(run) + + assert not isinstance(messages, BaseException), messages + assert server.calls == ["before", "secure"] + assert len(client.calls) == 1 + results = _function_results(messages) + assert all(len(items) == 1 for items in results.values()) + assert set(results) == {"call-0", "call-1", "call-2"} + assert results["call-1"][0].result == OAUTH_CONSENT_REQUIRED_RESULT + assert results["call-2"][0].result == OAUTH_CONSENT_NOT_EXECUTED_RESULT + assert "before-result" in str(results["call-0"][0].result) + consents = [c for c in _all_contents(messages) if c.type == "oauth_consent_request"] + assert [c.consent_link for c in consents] == [CONSENT_LINK] + assert scope.consent_required + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_parallel_function_loop_preserves_completed_siblings(stream: bool) -> None: + server = ToolboxConsentServer(["secure", "open"]) + server.require_consent("secure") + async with consent_toolbox(server) as toolbox: + client = BatchToolCallClient([function.name for function in toolbox.functions]) + agent = Agent(client=client, tools=toolbox, default_options=ChatOptions(allow_multiple_tool_calls=True)) + + async def run() -> list[Message]: + if stream: + response_stream = agent.run("go", stream=True) + async for _ in response_stream: + pass + return list((await response_stream.get_final_response()).messages) + return list((await agent.run("go")).messages) + + _, messages = await _in_scope(run) + + assert not isinstance(messages, BaseException), messages + assert len(client.calls) == 1 + results = _function_results(messages) + assert set(results) == {"call-0", "call-1"} + assert all(len(items) == 1 for items in results.values()) + assert results["call-0"][0].result == OAUTH_CONSENT_REQUIRED_RESULT + # Concurrent siblings may already have started; their completed or not-executed result is preserved. + assert results["call-1"][0].result == OAUTH_CONSENT_NOT_EXECUTED_RESULT or "open-result" in str( + results["call-1"][0].result + ) + + +# endregion + +# region Responses translation + + +async def test_tracker_emits_terminal_output_then_consent_and_marks_response_incomplete() -> None: + stream = ResponseEventStream(response_id="resp_consent") + stream.emit_created() + stream.emit_in_progress() + tracker = _OutputItemTracker( + stream, allowed_oauth_consent_origins=_normalize_allowed_oauth_consent_origins(["https://consent.example.com"]) + ) + contents = [ + Content.from_function_call(call_id="call-1", name="secure", arguments="{}"), + Content.from_function_result(call_id="call-1", result=OAUTH_CONSENT_REQUIRED_RESULT), + Content.from_oauth_consent_request(CONSENT_LINK, additional_properties={"server_label": "secure-box"}), + Content.from_function_call(call_id="call-2", name="other", arguments="{}"), + Content.from_function_result(call_id="call-2", result=OAUTH_CONSENT_NOT_EXECUTED_RESULT), + ] + events: list[Any] = [] + for content in contents: + async for event in tracker.handle(content): + events.append(event) + events.extend(tracker.close()) + + done_items = [ + dict(e["item"]) if isinstance(e, Mapping) else e.item.as_dict() + for e in events + if (e.get("type") if isinstance(e, Mapping) else e.type) == "response.output_item.done" + ] + assert [item["type"] for item in done_items] == [ + "function_call", + "function_call_output", + "oauth_consent_request", + "function_call", + "function_call_output", + ] + outputs = {item["call_id"]: item["output"] for item in done_items if item["type"] == "function_call_output"} + assert outputs == {"call-1": OAUTH_CONSENT_REQUIRED_RESULT, "call-2": OAUTH_CONSENT_NOT_EXECUTED_RESULT} + consent = next(item for item in done_items if item["type"] == "oauth_consent_request") + assert consent["consent_link"] == CONSENT_LINK + assert consent["server_label"] == "secure-box" + assert tracker.oauth_consent_requested + + +async def test_tracker_rejects_consent_from_disallowed_origin() -> None: + stream = ResponseEventStream(response_id="resp_consent_rejected") + stream.emit_created() + tracker = _OutputItemTracker( + stream, allowed_oauth_consent_origins=_normalize_allowed_oauth_consent_origins(["https://other.example.com"]) + ) + with pytest.raises(ValueError, match="allowed safe HTTPS"): + async for _ in tracker.handle(Content.from_oauth_consent_request(CONSENT_LINK)): + pass + + +# endregion diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_options_sample.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_options_sample.py new file mode 100644 index 00000000000..fed1df412cb --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_options_sample.py @@ -0,0 +1,77 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Exercise the documented option hook through the real Responses host offline.""" + +from __future__ import annotations + +import importlib.util +from copy import deepcopy +from pathlib import Path +from types import ModuleType +from typing import Any +from unittest.mock import patch + +import httpx +import pytest +from agent_framework import Agent, ChatOptions + +from agent_framework_foundry_hosting import ResponsesHostServer + +from .options_helpers import OptionsClient + + +@pytest.fixture(scope="module") +def options_sample() -> ModuleType: + path = ( + Path(__file__).resolve().parents[4] + / "samples" + / "04-hosting" + / "foundry-hosted-agents" + / "responses" + / "basic" + / "options.py" + ) + spec = importlib.util.spec_from_file_location("foundry_responses_options_sample", path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + with patch("dotenv.load_dotenv", return_value=False): + spec.loader.exec_module(module) + return module + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("nested", [False, True]) +async def test_sample_hook_preserves_developer_model_and_other_caller_options( + options_sample: ModuleType, stream: bool, nested: bool +) -> None: + model = OptionsClient() + defaults: ChatOptions = {"model": "developer", "temperature": 0.4, "max_tokens": 256} + agent = Agent(client=model, default_options=defaults) + original_defaults = deepcopy(agent.default_options) + host = ResponsesHostServer(agent, prepare_options=options_sample.prepare_options) + payload: dict[str, Any] = { + "input": "hi", + "model": "caller", + "temperature": 0.8, + "max_output_tokens": 300, + "max_tokens": 200, + "stream": stream, + "store": False, + } + if nested: + payload["extra_body"] = {"model": "nested-caller", "temperature": 0.6, "max_tokens": 150} + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/responses", json=payload) + + assert response.status_code == 200 + assert "response.failed" not in response.text + if stream: + assert "event: response.completed" in response.text + else: + assert response.json()["status"] == "completed" + assert len(model.calls) == 1 + assert model.calls[0]["model"] == "developer" + assert model.calls[0]["temperature"] == (0.6 if nested else 0.8) + assert model.calls[0]["max_tokens"] == (150 if nested else 200) + assert agent.default_options == original_defaults + assert defaults == {"model": "developer", "temperature": 0.4, "max_tokens": 256} diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_session_approvals.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_session_approvals.py new file mode 100644 index 00000000000..a2b4a80ee90 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_session_approvals.py @@ -0,0 +1,101 @@ +# Copyright (c) Microsoft. All rights reserved. + +from __future__ import annotations + +import pytest +from agent_framework import AgentSession, Content + +from agent_framework_foundry_hosting._approvals import _APPROVAL_STATE_KEY, SessionApprovalState, _InvalidApproval + +from .approval_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + SqliteSessionStore, + approval_request, + pending_session, +) + + +async def test_save_and_load_roundtrip_in_session_state() -> None: + session = AgentSession() + state = SessionApprovalState(session) + await state.save_approval_request("wire", approval_request()) + assert set(session.state[_APPROVAL_STATE_KEY]) == {"wire"} + assert await state.load_approval_request("wire") == approval_request() + + +async def test_requests_persist_through_session_store_without_type_rehydration(tmp_path) -> None: + store = SqliteSessionStore(tmp_path / "sessions.db") + await pending_session(store) + for alias in ("response-alias", "conversation-alias"): + session = await store.get(alias) + assert session is not None + content = await SessionApprovalState(session).load_approval_request("wire-0") + assert content == approval_request() + assert content.function_call is not None + assert content.function_call.hosted is True + assert content.function_call.additional_properties["provider"] == {"type": "message", "opaque_id": "original"} + + +async def test_loading_does_not_consume_so_forks_and_retries_see_the_same_request() -> None: + session = AgentSession() + await SessionApprovalState(session).save_approval_request("wire", approval_request()) + fork = AgentSession.from_dict(session.to_dict()) + for state in (SessionApprovalState(session), SessionApprovalState(fork)): + assert await state.load_approval_request("wire") == approval_request() + assert await state.load_approval_request("wire") == approval_request() + + +async def test_unknown_wire_id_raises_invalid_approval() -> None: + state = SessionApprovalState(AgentSession()) + with pytest.raises(_InvalidApproval, match="not in this session"): + await state.load_approval_request("missing") + assert issubclass(_InvalidApproval, ValueError) + + +@pytest.mark.parametrize("wire_id", ["", None]) +async def test_save_rejects_empty_wire_id(wire_id) -> None: + session = AgentSession() + with pytest.raises(ValueError, match="non-empty string"): + await SessionApprovalState(session).save_approval_request(wire_id, approval_request()) + assert _APPROVAL_STATE_KEY not in session.state + + +async def test_save_rejects_duplicate_wire_id_without_overwriting() -> None: + session = AgentSession() + state = SessionApprovalState(session) + await state.save_approval_request("wire", approval_request(0)) + with pytest.raises(ValueError, match="already exists"): + await state.save_approval_request("wire", approval_request(1)) + assert await state.load_approval_request("wire") == approval_request(0) + + +@pytest.mark.parametrize( + "request_content", + [ + Content.from_text("not an approval"), + Content.from_function_approval_request("approval", Content.from_function_call(call_id="", name="tool")), + ], +) +async def test_save_rejects_incomplete_requests(request_content) -> None: + session = AgentSession() + with pytest.raises(ValueError, match="complete function approval requests"): + await SessionApprovalState(session).save_approval_request("wire", request_content) + assert _APPROVAL_STATE_KEY not in session.state + + +@pytest.mark.parametrize( + "stored", + [ + [], + {"wire": 1}, + {"": "{}"}, + {"wire": "not json"}, + {"wire": "[]"}, + {"wire": '{"type":"text","text":"x"}'}, + ], +) +async def test_malformed_stored_map_fails_closed(stored) -> None: + session = AgentSession() + session.state[_APPROVAL_STATE_KEY] = stored + state = SessionApprovalState(session) + with pytest.raises(RuntimeError, match="start a fresh conversation"): + await state.load_approval_request("wire") diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py index 0a9896bb218..4e3b7a333a7 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py @@ -28,9 +28,9 @@ WorkflowInvocationKwargs, handler, response_handler, - tool, ) from agent_framework._workflows._const import RESOLVED_WORKFLOW_RUN_KWARGS_KEY +from agent_framework.exceptions import AgentInvalidResponseException from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext from azure.ai.agentserver.core.storage import FoundryStorageConflictError, FoundryStoragePreconditionError @@ -44,6 +44,15 @@ WorkflowHead, ) +from .workflow_tool_helpers import ( + MutableToolOwner, + UnusedClient, + mutable_function_tool, + shared_async_function, + shared_function, + workflow_with_tools, +) + def _scope(*, user: str = "user", sandbox: str = "sandbox", call: str = "call") -> FoundryRequestScope: return FoundryRequestScope(sandbox, user, call, False) @@ -106,6 +115,25 @@ async def decide(self, request: Content, response: Content, ctx: WorkflowContext await ctx.yield_output("approved" if response.approved else "rejected") +class _Computer(Executor): + def __init__(self) -> None: + super().__init__("computer") + + @handler + async def start(self, message: str, ctx: WorkflowContext[str, str]) -> None: + call = Content.from_computer_tool_call( + id="item-1", + call_id="computer-1", + actions=[{"type": "screenshot"}], + pending_safety_checks=[{"id": "check-1", "code": "risk", "message": message}], + ) + await ctx.request_info(call, Content, request_id="computer-1") + + @response_handler + async def done(self, request: Content, response: Content, ctx: WorkflowContext[str, str]) -> None: + await ctx.yield_output(f"computer:{response.call_id}") + + class _Loop(Executor): def __init__(self, calls: list[str] | None = None) -> None: super().__init__("loop") @@ -263,30 +291,65 @@ async def create(request: object) -> Workflow: assert requests == [one, two] -@pytest.mark.parametrize("mcp", [False, True]) -async def test_resolver_rejects_reused_local_and_mcp_tools(mcp: bool) -> None: - class _UnusedClient(BaseChatClient): - async def _inner_get_response(self, **kwargs: Any) -> Any: - raise AssertionError("Ownership validation must not call the model.") - - @tool - def local_tool(value: str) -> str: - """Return a value without accessing external services.""" - return value - - shared = MCPStreamableHTTPTool(name="shared", url="https://unused.example.test/mcp") if mcp else local_tool +@pytest.mark.parametrize("shared", [shared_function, shared_async_function]) +async def test_resolver_allows_reused_stateless_function_tools(shared: Any) -> None: + resolver: WorkflowResolver[object] = WorkflowResolver(lambda request: workflow_with_tools([shared])) + first, second = await resolver.resolve(object()), await resolver.resolve(object()) + assert first is not second + for workflow in (first, second): + executor = workflow.get_start_executor() + assert isinstance(executor, AgentExecutor) + assert isinstance(executor.agent, RawAgent) + assert executor.agent.default_options["tools"][0] is shared - def create(request: object) -> Workflow: - agent = Agent(client=_UnusedClient(), name="agent", tools=[shared]) - return _workflow(AgentExecutor(agent, id="agent")) - resolver: WorkflowResolver[object] = WorkflowResolver(create) +async def test_resolver_rejects_reused_mcp_tools() -> None: + shared = MCPStreamableHTTPTool(name="shared", url="https://unused.example.test/mcp") + resolver: WorkflowResolver[object] = WorkflowResolver( + lambda request: workflow_with_tools([shared_function, shared]) + ) first = await resolver.resolve(object()) with pytest.raises(RuntimeError, match="cannot share"): await resolver.resolve(object()) assert first.get_start_executor().id == "agent" +@pytest.mark.parametrize( + "kind", ["bound", "closure", "subclass", "parser", "max_invocations", "max_invocation_exceptions"] +) +async def test_resolver_rejects_reused_mutable_tools_but_allows_fresh_ones(kind: str) -> None: + shared = mutable_function_tool(kind) + resolver: WorkflowResolver[object] = WorkflowResolver( + lambda request: workflow_with_tools([shared_function, shared]) + ) + await resolver.resolve(object()) + with pytest.raises(RuntimeError, match="mutable tools"): + await resolver.resolve(object()) + fresh: WorkflowResolver[object] = WorkflowResolver( + lambda request: workflow_with_tools([shared_function, mutable_function_tool(kind)]) + ) + await fresh.resolve(object()) + await fresh.resolve(object()) + + +async def test_resolver_rejects_reused_owner_even_with_fresh_bound_tool_wrappers() -> None: + owner = MutableToolOwner() + resolver: WorkflowResolver[object] = WorkflowResolver(lambda request: workflow_with_tools([owner.count])) + await resolver.resolve(object()) + with pytest.raises(RuntimeError, match="cannot share"): + await resolver.resolve(object()) + + +async def test_resolver_rejects_reused_client_with_stateless_tools() -> None: + client = UnusedClient() + resolver: WorkflowResolver[object] = WorkflowResolver( + lambda request: workflow_with_tools([shared_function], client=client) + ) + await resolver.resolve(object()) + with pytest.raises(RuntimeError, match="cannot share"): + await resolver.resolve(object()) + + async def test_resolver_rejects_reused_registry_backed_agent_resources() -> None: class _UnusedClient(BaseChatClient): async def _inner_get_response(self, **kwargs: Any) -> Any: @@ -371,14 +434,10 @@ def create(request: object) -> Workflow: resolver = WorkflowResolver(create) workflow = await resolver.resolve(object()) run = await _prepare("model-turn", workflow=workflow, fresh_factory=True) - turn = run.validate_turn(WorkflowTurn(input=[Message("user", "hello")])) - clients, functions = prepare_workflow_kwargs( - workflow, - turn, - run.scope, - options={"temperature": 0.6}, - fresh_factory=True, + turn = run.validate_turn( + WorkflowTurn(input=[Message("user", "hello")], client_kwargs={"options": {"temperature": 0.6}}) ) + clients, functions = prepare_workflow_kwargs(workflow, turn, run.scope, fresh_factory=True) await run.claim() events = [event async for event in run.events(turn, client_kwargs=clients, function_invocation_kwargs=functions)] assert any(event.type == "output" for event in events) @@ -401,9 +460,8 @@ def create(request: object) -> Workflow: id="model", ) ), - WorkflowTurn(input=[Message("user", "hello")]), + WorkflowTurn(input=[Message("user", "hello")], client_kwargs={"options": {"temperature": 0.6}}), run.scope, - options={"temperature": 0.6}, fresh_factory=True, ) @@ -503,42 +561,70 @@ async def test_graph_and_name_mismatch(changed: Workflow) -> None: await _prepare("two", workflow=changed, lineage_id="invocations") -async def test_partial_and_invalid_replies_do_not_claim_or_consume() -> None: +async def test_invalid_replies_fail_before_superstep_and_release_the_head() -> None: first = await _prepare("one", workflow=_workflow(_Review())) await _finish(first, WorkflowTurn(input="review")) - second = await _prepare("two", workflow=_workflow(_Review()), previous_response_id="one") - for replies in ({"first": True}, {"first": True, "forged": False}, {"first": {}, "second": True}): + for index, replies in enumerate(({"first": True, "forged": False}, {"first": {}, "second": True})): + attempt = await _prepare(f"bad-{index}", workflow=_workflow(_Review()), previous_response_id="one") + turn = attempt.validate_turn(WorkflowTurn(responses=replies)) + await attempt.claim() with pytest.raises(ValueError): - second.validate_turn(WorkflowTurn(responses=replies)) - head, _ = await second.store.get_head("one", None) - assert head is not None and head.response_id is None - assert await _finish(second, WorkflowTurn(responses={"first": True, "second": False})) == [ - "review:True", - "review:False", - ] + async for _ in attempt.events(turn): + pass + await attempt.abort() + head, _ = await attempt.store.get_head("one", None) + assert head is not None and head.response_id is None and not head.blocked + assert await attempt.store.get_response(f"bad-{index}") == (None, None) + + partial = await _prepare("partial", workflow=_workflow(_Review()), previous_response_id="one") + assert await _finish(partial, WorkflowTurn(responses={"first": True})) == ["review:True"] + rest = await _prepare("rest", workflow=_workflow(_Review()), previous_response_id="partial") + assert set(rest.pending_requests) == {"second"} + assert await _finish(rest, WorkflowTurn(responses={"second": False})) == ["review:False"] with pytest.raises(WorkflowConflictError, match="stale or forked"): await _prepare("replay", workflow=_workflow(_Review()), previous_response_id="one") -async def test_forged_embedded_approval_is_rejected_before_consumption() -> None: - first = await _prepare("one", workflow=_workflow(_Approval())) - await _finish(first, WorkflowTurn(input="/safe")) - second = await _prepare("two", workflow=_workflow(_Approval()), previous_response_id="one") - pending = second.pending_requests["approval-1"].data - forged = Content.from_function_call("call-1", "delete", arguments={"path": "/forged"}, id="approval-1") - with pytest.raises(ValueError, match="exact pending"): - second.validate_turn( - WorkflowTurn( - responses={ - "approval-1": Content.from_function_approval_response(True, "approval-1", forged), - } - ) - ) - decision = pending.to_function_approval_response(True) - assert await _finish(second, WorkflowTurn(responses={"approval-1": decision})) == ["approved"] +async def test_unacknowledged_computer_result_releases_the_head_for_a_valid_retry() -> None: + first = await _prepare("one", workflow=_workflow(_Computer())) + await _finish(first, WorkflowTurn(input="review")) + unacknowledged = Content.from_computer_tool_result(call_id="computer-1") + attempt = await _prepare("bad", workflow=_workflow(_Computer()), previous_response_id="one") + turn = attempt.validate_turn(WorkflowTurn(responses={"computer-1": unacknowledged})) + await attempt.claim() + with pytest.raises(AgentInvalidResponseException, match="safety checks"): + async for _ in attempt.events(turn): + pass + await attempt.abort() + head, _ = await attempt.store.get_head("one", None) + assert head is not None and head.response_id is None and not head.blocked + + acknowledged = Content.from_computer_tool_result( + call_id="computer-1", acknowledged_safety_checks=[{"id": "check-1", "code": "risk", "message": "review"}] + ) + retry = await _prepare("retry", workflow=_workflow(_Computer()), previous_response_id="one") + assert await _finish(retry, WorkflowTurn(responses={"computer-1": acknowledged})) == ["computer:computer-1"] -async def test_racing_claims_consume_authority_once() -> None: +async def test_failure_after_first_superstep_still_blocks_the_lineage() -> None: + first = await _prepare("one", workflow=_workflow(_Review())) + await _finish(first, WorkflowTurn(input="review")) + attempt = await _prepare("bad", workflow=_workflow(_Review()), previous_response_id="one") + turn = attempt.validate_turn(WorkflowTurn(responses={"first": True, "second": False})) + await attempt.claim() + events = attempt.events(turn) + async for event in events: + if event.type == "superstep_started": + break + await events.aclose() + await attempt.abort() + head, _ = await attempt.store.get_head("one", None) + assert head is not None and head.blocked + with pytest.raises(WorkflowBlockedError): + await _prepare("retry", workflow=_workflow(_Review()), previous_response_id="one") + + +async def test_racing_claims_admit_one_and_pre_superstep_abort_releases() -> None: first = await _prepare("one") await _finish(first, WorkflowTurn(input=1)) left = await _prepare("left", previous_response_id="one") @@ -549,8 +635,8 @@ async def test_racing_claims_consume_authority_once() -> None: with pytest.raises(RuntimeError, match="Another request"): await right.claim() await left.abort() - with pytest.raises(RuntimeError, match="fresh workflow lineage"): - await _prepare("retry", previous_response_id="one") + retry = await _prepare("retry", previous_response_id="one") + assert await _finish(retry, WorkflowTurn(input=1)) == [2] @pytest.mark.parametrize("error", [FoundryStorageConflictError, FoundryStoragePreconditionError]) @@ -668,6 +754,8 @@ async def test_checkpoint_mutation_and_foreign_pairing_fail() -> None: {"session": "forged"}, {"additional_function_arguments": {"call_id": "forged"}}, {"options": {"extra_body": {"store": True}}}, + {"compaction_strategy": object()}, + {"options": {"compaction_strategy": object()}}, WorkflowInvocationKwargs(executor_kwargs={"counter": {"options": {"instructions": "forged"}}}), ], ) @@ -679,9 +767,12 @@ def test_client_kwargs_reject_controls_in_all_namespaces(kwargs: Any) -> None: def test_tool_context_stays_separate_from_model_options() -> None: clients, functions = prepare_workflow_kwargs( _workflow(), - WorkflowTurn(input=1, function_invocation_kwargs={"ticket": "application"}), + WorkflowTurn( + input=1, + client_kwargs={"options": {"temperature": 0.5}}, + function_invocation_kwargs={"ticket": "application"}, + ), _scope(), - options={"temperature": 0.5}, ) assert functions.global_kwargs == {"ticket": "application"} assert clients.global_kwargs["additional_function_arguments"] == { diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py index 81fd0550e76..c48c2a1fcae 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py @@ -18,10 +18,22 @@ FunctionInvocationLayer, Message, ResponseStream, + SessionStore, ) +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext from azure.core.credentials import AccessToken -from agent_framework_foundry_hosting import FoundryToolbox +from agent_framework_foundry_hosting import FoundryToolbox, StoreProvider + + +class FixedSessionStoreProvider(StoreProvider[SessionStore]): + """Return one application-owned session store for every request.""" + + def __init__(self, store: SessionStore) -> None: + self.store = store + + def get_store(self, *, config: AgentConfig, platform_context: FoundryAgentRequestContext) -> SessionStore: + return self.store class ToolboxCredential: diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_consent_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_consent_helpers.py new file mode 100644 index 00000000000..71bb3c3afd7 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_consent_helpers.py @@ -0,0 +1,140 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Offline protocol boundaries for invocation-time Foundry Toolbox OAuth consent tests.""" + +from __future__ import annotations + +import json +from collections.abc import AsyncGenerator, AsyncIterator, Awaitable, Mapping, Sequence +from contextlib import asynccontextmanager +from typing import Any + +import httpx2 as httpx +from agent_framework import ( + BaseChatClient, + ChatResponse, + ChatResponseUpdate, + Content, + FunctionInvocationLayer, + Message, + ResponseStream, +) +from azure.core.credentials import AccessToken + +from agent_framework_foundry_hosting import FoundryToolbox + +CONSENT_LINK = "https://consent.example.com/authorize?state=abc" + + +class ConsentCredential: + def get_token(self, *scopes: str, **kwargs: Any) -> AccessToken: + return AccessToken("offline-token", 9999999999) + + +class ToolboxConsentServer: + """Offline MCP server whose listed tools succeed unless configured to return a ``-32006`` consent error.""" + + def __init__(self, tool_names: Sequence[str]) -> None: + self.tool_names = list(tool_names) + self.consent_errors: dict[str, dict[str, Any]] = {} + self.calls: list[str] = [] + + def require_consent( + self, + tool_name: str, + *, + message: str = f"Request failed (remote): {CONSENT_LINK}", + data: Any = None, + ) -> None: + error: dict[str, Any] = {"code": -32006, "message": message} + if data is not None: + error["data"] = data + self.consent_errors[tool_name] = error + + async def handle(self, request: httpx.Request) -> httpx.Response: + if request.method == "GET": + return httpx.Response(405) + if request.method == "DELETE": + return httpx.Response(200) + payload = json.loads(request.content) + method = payload.get("method") + if "id" not in payload: + return httpx.Response(202) + result: dict[str, Any] + if method == "server/discover": + return httpx.Response( + 200, + json={"jsonrpc": "2.0", "id": payload["id"], "error": {"code": -32601, "message": "Method not found"}}, + ) + if method == "initialize": + result = { + "protocolVersion": payload["params"]["protocolVersion"], + "capabilities": {"tools": {}}, + "serverInfo": {"name": "toolbox-consent", "version": "1"}, + } + elif method == "tools/list": + result = { + "tools": [ + {"name": name, "inputSchema": {"type": "object", "properties": {}}} for name in self.tool_names + ] + } + elif method == "tools/call": + name = payload["params"]["name"] + self.calls.append(name) + if (error := self.consent_errors.get(name)) is not None: + return httpx.Response(200, json={"jsonrpc": "2.0", "id": payload["id"], "error": error}) + result = {"content": [{"type": "text", "text": f"{name}-result"}]} + elif method == "ping": + result = {} + else: + raise AssertionError(f"Unexpected MCP method: {method}") + return httpx.Response(200, json={"jsonrpc": "2.0", "id": payload["id"], "result": result}) + + +@asynccontextmanager +async def consent_toolbox(server: ToolboxConsentServer, **kwargs: Any) -> AsyncGenerator[FoundryToolbox]: + toolbox = FoundryToolbox(ConsentCredential(), url="https://toolbox.invalid/toolboxes/secure/mcp", **kwargs) + original = toolbox._httpx_client + assert original is not None + await original.aclose() + toolbox._httpx_client = httpx.AsyncClient(transport=httpx.MockTransport(server.handle)) + try: + await toolbox.connect() + yield toolbox + finally: + await toolbox.close() + + +class BatchToolCallClient(FunctionInvocationLayer[Any], BaseChatClient[Any]): + """Requests one batch of function calls on its first turn, then answers with text.""" + + def __init__(self, tool_names: Sequence[str]) -> None: + super().__init__() + self.tool_names = list(tool_names) + self.calls: list[list[Message]] = [] + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + stream: bool, + options: Mapping[str, Any], + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + self.calls.append(list(messages)) + contents = ( + [ + Content.from_function_call(call_id=f"call-{index}", name=name, arguments="{}") + for index, name in enumerate(self.tool_names) + ] + if len(self.calls) == 1 + else [Content.from_text("done")] + ) + + async def updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate(role="assistant", contents=contents) + + async def complete() -> ChatResponse: + return ChatResponse(messages=[Message("assistant", contents=contents)]) + + return ResponseStream(updates(), finalizer=ChatResponse.from_updates) if stream else complete() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/workflow_tool_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/workflow_tool_helpers.py new file mode 100644 index 00000000000..9de39007638 --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/workflow_tool_helpers.py @@ -0,0 +1,70 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Model-free tools and workflow factories for resource-ownership tests.""" + +from __future__ import annotations + +from typing import Any + +from agent_framework import Agent, AgentExecutor, BaseChatClient, FunctionTool, Workflow, WorkflowBuilder, tool + + +class UnusedClient(BaseChatClient): + async def _inner_get_response(self, **kwargs: Any) -> Any: + raise AssertionError("Ownership validation must not call the model.") + + +@tool +def shared_function(value: str) -> str: + """Return a value without accessing external services.""" + return value + + +@tool +async def shared_async_function(value: str) -> str: + """Return a value asynchronously without accessing external services.""" + return value + + +class MutableToolOwner: + def __init__(self) -> None: + self.calls = 0 + + @tool + def count(self, value: str) -> str: + """Count calls and return the input.""" + self.calls += 1 + return value + + +class CustomFunctionTool(FunctionTool): + pass + + +def mutable_function_tool(kind: str) -> FunctionTool: + if kind == "bound": + return MutableToolOwner().count + if kind == "closure": + calls: list[str] = [] + + @tool + def capture(value: str) -> str: + """Record and return the input.""" + calls.append(value) + return value + + return capture + if kind == "subclass": + return CustomFunctionTool(name="custom", func=shared_function.func) + if kind == "parser": + return FunctionTool(name="parser", func=shared_function.func, result_parser=MutableToolOwner().count) + if kind == "max_invocations": + return FunctionTool(name="limited", func=shared_function.func, max_invocations=1) + if kind == "max_invocation_exceptions": + return FunctionTool(name="limited", func=shared_function.func, max_invocation_exceptions=1) + raise ValueError(f"Unknown mutable tool kind: {kind}") + + +def workflow_with_tools(tools: list[Any], *, client: BaseChatClient | None = None) -> Workflow: + agent = Agent(client=client or UnusedClient(), name="agent", tools=tools) + return WorkflowBuilder(start_executor=AgentExecutor(agent, id="agent")).build() diff --git a/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py b/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py index e3aa46d553f..1d503568a9c 100644 --- a/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py +++ b/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py @@ -6,7 +6,7 @@ import subprocess import sys import textwrap -from typing import Any, cast +from typing import Any from unittest.mock import patch import pytest @@ -94,17 +94,13 @@ def test_host_signatures_expose_only_the_selected_execution_model(protocol: str) assert workflow_parameters["workflow"].default is inspect.Parameter.empty assert workflow_parameters[parser].default is inspect.Parameter.empty assert workflow_parameters[parser].kind is inspect.Parameter.KEYWORD_ONLY - for legacy in ("parse_response", "parse_request"): - assert legacy not in agent_parameters - assert legacy not in workflow_parameters assert "checkpoint_store_provider" in workflow_parameters assert "checkpoint_store_provider" not in agent_parameters - for name in ("agent", "agent_session_store_provider", "history_source", "background_source", "unsupported_options"): + for name in ("agent", "agent_session_store_provider", "history_source", "background_source", "prepare_options"): assert name not in workflow_parameters - assert ("prepare_options" in workflow_parameters) is (protocol == "Responses") + assert "prepare_options" in agent_parameters if protocol == "Responses": assert "response_store" in agent_parameters - assert "store" not in agent_parameters @pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) @@ -136,36 +132,15 @@ def test_agent_hosts_reject_ambiguous_factories_before_sdk_construction(protocol initialize.assert_not_called() +@pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) @pytest.mark.parametrize( - ("protocol", "setting"), - [ - ("Responses", "agent"), - ("Responses", "agent_session_store_provider"), - ("Responses", "function_approval_store_provider"), - ("Responses", "history_source"), - ("Responses", "background_source"), - ("Responses", "unsupported_options"), - ("Responses", "store"), - ("Invocations", "agent"), - ("Invocations", "agent_session_store_provider"), - ("Invocations", "prepare_options"), - ("Invocations", "unsupported_options"), - ("Invocations", "legacy_wire_format"), - ], + "setting", + ["agent_session_store_provider", "history_source", "background_source", "max_history_items", "prepare_options"], ) -def test_workflow_hosts_reject_agent_settings_before_sdk_construction(protocol: str, setting: str) -> None: +def test_workflow_hosts_do_not_accept_agent_settings(protocol: str, setting: str) -> None: workflow_host = getattr(hosting, f"{protocol}WorkflowHostServer") - parser = "parse_input" - values: dict[str, Any] = {parser: lambda _: None, setting: None} - base = ( - "azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost" - if protocol == "Responses" - else "azure.ai.agentserver.invocations.InvocationAgentServerHost" - ) - with patch(f"{base}.__init__", return_value=None) as initialize: - with pytest.raises(TypeError, match=setting): - workflow_host(workflow=lambda _: None, **values) - initialize.assert_not_called() + with pytest.raises(TypeError, match=setting): + workflow_host(workflow=lambda request: None, parse_input=lambda _: None, **{setting: None}) @pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) @@ -176,44 +151,3 @@ def test_workflow_hosts_validate_factories_and_parsers_at_construction(protocol: workflow_host(workflow=lambda _: None, **{parser: None}) with pytest.raises(TypeError, match="one request"): workflow_host(workflow=lambda: None, **{parser: lambda _: None}) - - -def test_legacy_responses_request_name_is_not_exported() -> None: - assert "HostedResponseRequest" not in hosting.__all__ - assert "HostedResponseRequest" not in dir(hosting) - with pytest.raises(AttributeError, match="HostedResponseRequest"): - hosting.__getattr__("HostedResponseRequest") - - -@pytest.mark.parametrize( - ("name", "legacy"), - [ - ("ResponsesWorkflowHostServer", "parse_response"), - ("InvocationsHostServer", "parse_request"), - ("InvocationsWorkflowHostServer", "parse_request"), - ], -) -def test_legacy_parser_names_are_not_silently_forwarded_to_sdk(name: str, legacy: str) -> None: - host = getattr(hosting, name) - source = "workflow" if "Workflow" in name else "agent" - values = {source: lambda _: None, "parse_input": lambda _: None, legacy: lambda _: None} - base = ( - "azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost" - if name.startswith("Responses") - else "azure.ai.agentserver.invocations.InvocationAgentServerHost" - ) - with patch(f"{base}.__init__", return_value=None) as initialize: - with pytest.raises(TypeError, match=f"{legacy} was renamed to parse_input"): - host(**values) - initialize.assert_not_called() - - -def test_responses_workflow_host_preserves_steering_guard() -> None: - from azure.ai.agentserver.responses import ResponsesServerOptions - - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable"): - cast(Any, hosting.ResponsesWorkflowHostServer)( - workflow=lambda _: None, - parse_input=lambda _: None, - options=ResponsesServerOptions(steerable_conversations=True), - ) diff --git a/python/packages/foundry_hosting/tests/test_invocations.py b/python/packages/foundry_hosting/tests/test_invocations.py index 3a488f95365..5f276d3dd83 100644 --- a/python/packages/foundry_hosting/tests/test_invocations.py +++ b/python/packages/foundry_hosting/tests/test_invocations.py @@ -13,13 +13,12 @@ import asyncio import json -import logging import uuid import weakref from collections.abc import AsyncIterator, Awaitable, Callable, Generator, Mapping, Sequence from contextlib import contextmanager from itertools import product -from typing import Any, Literal, cast +from typing import Any, cast from unittest.mock import AsyncMock, MagicMock, patch import httpx @@ -302,7 +301,7 @@ async def echo(self, messages: list[Message], ctx: WorkflowContext[str, str]) -> def _make_workflow_agent() -> WorkflowAgent: - return WorkflowBuilder(name="legacy", start_executor=_EchoExecutor()).build().as_agent() + return WorkflowBuilder(name="wrapped", start_executor=_EchoExecutor()).build().as_agent() async def _invoke_text(server: InvocationsHostServer, message: str) -> str: @@ -667,8 +666,6 @@ def create_agent(request: Request, name: str) -> _FakeAgent: [ ("parse_input", 42, "parse_input must be a callable"), ("prepare_options", 42, "prepare_options must be a callable"), - ("unsupported_options", "discard", "unsupported_options"), - ("legacy_wire_format", "false", "legacy_wire_format must be a boolean"), ], ) def test_rejects_invalid_configuration(self, parameter: str, value: Any, error: str) -> None: @@ -828,7 +825,7 @@ def broken_parser(_request: Request) -> InvocationRun: ) async def test_reserved_options_are_rejected_before_agent_run(self, options: dict[str, Any]) -> None: agent = _make_agent(response_text="ok") - server = InvocationsHostServer(agent) + server = InvocationsHostServer(agent, prepare_options=lambda _request, options: options) with _request_context(session_id="options"): response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] _make_request({"message": "hi", "options": options}) @@ -871,7 +868,7 @@ def create_agent(*, request: Request) -> _FakeAgent: parse_input=(lambda _request: InvocationRun(messages="hi", options=options, stream=stream)) if source == "parser" else None, - prepare_options=(lambda _request, _options: options) if source == "hook" else None, + prepare_options=lambda _request, parsed_options: options if source == "hook" else parsed_options, ) with _request_context(session_id="unsafe-options"): response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] @@ -887,11 +884,7 @@ def create_agent(*, request: Request) -> _FakeAgent: assert provider.contexts == [] @pytest.mark.parametrize("stream", [False, True]) - @pytest.mark.parametrize("legacy_wire_format", [False, True]) - @pytest.mark.parametrize("policy", ["ignore", "warn", "error"]) - async def test_http_route_rejects_tool_context_options_under_every_wire_and_options_policy( - self, stream: bool, legacy_wire_format: bool, policy: Literal["ignore", "warn", "error"] - ) -> None: + async def test_http_route_rejects_tool_context_options(self, stream: bool) -> None: agent = _make_agent(response_text="ok", stream_texts=["ok"]) provider = _SessionStoreProvider(_mock_session_store()) agent.default_options = {"additional_function_arguments": {"user_id": "trusted", "tenant_id": "trusted"}} @@ -900,15 +893,10 @@ def create_host() -> InvocationsHostServer: return InvocationsHostServer( agent, agent_session_store_provider=provider, - unsupported_options=policy, - legacy_wire_format=legacy_wire_format, + prepare_options=lambda _request, options: options, ) - if legacy_wire_format: - with pytest.warns(DeprecationWarning, match="legacy_wire_format"): - server = create_host() - else: - server = create_host() + server = create_host() async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: response = await client.post( "/invocations", @@ -940,33 +928,36 @@ async def test_options_hook_must_return_safe_string_keyed_mapping(self, result: assert agent.calls == [] @pytest.mark.parametrize("stream", [False, True]) - @pytest.mark.parametrize("policy", ["ignore", "warn", "error"]) - async def test_unsupported_options_policy( - self, policy: Literal["ignore", "warn", "error"], stream: bool, caplog: pytest.LogCaptureFixture - ) -> None: + async def test_hook_options_for_agent_without_options_are_rejected_before_streaming(self, stream: bool) -> None: agent = _NoOptionsAgent() - server = InvocationsHostServer(agent, unsupported_options=policy) - with _request_context(session_id="unsupported"): + server = InvocationsHostServer(agent, prepare_options=lambda _request, options: options) + with _request_context(session_id="no-options"): response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] _make_request({"message": "hello", "options": {"temperature": 0.8}, "stream": stream}) ) - if policy == "error": - assert response.status_code == 400 - assert not isinstance(response, StreamingResponse) - assert "does not accept" in json.loads(bytes(response.body))["error"] - assert agent.calls == [] - else: - assert await _success_text(response) == "ok" - assert agent.calls[-1]["options"] is None + assert response.status_code == 400 + assert not isinstance(response, StreamingResponse) + assert "does not accept" in json.loads(bytes(response.body))["error"] + assert agent.calls == [] - assert ("Agent doesn't support runtime options" in caplog.text) is (policy == "warn") + async def test_caller_options_are_not_forwarded_without_hook(self) -> None: + agent = _make_agent(response_text="ok") + agent.default_options = {"temperature": 0.1} + server = InvocationsHostServer(agent) + with _request_context(session_id="defaults-only"): + response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] + _make_request({"message": "hello", "options": {"temperature": 0.9, "model": "other"}}) + ) + assert await _success_text(response) == "ok" + assert agent.calls[-1]["options"] is None + assert agent.default_options == {"temperature": 0.1} async def test_factory_options_failure_is_framed_before_first_delta(self) -> None: provider = _SessionStoreProvider(_mock_session_store()) server = InvocationsHostServer( lambda request: _NoOptionsAgent(), agent_session_store_provider=provider, - unsupported_options="error", + prepare_options=lambda _request, options: options, ) with _request_context(session_id="factory-options"): response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] @@ -977,8 +968,8 @@ async def test_factory_options_failure_is_framed_before_first_delta(self) -> Non ( "error", { - "message": "The hosted agent does not accept caller runtime options.", - "code": "unsupported_options", + "message": "The hosted agent does not accept the run options returned by prepare_options.", + "code": "invalid_request", "status": 400, }, ) @@ -1367,53 +1358,6 @@ async def test_invalid_json_uses_client_status_on_actual_http_route(self) -> Non assert "error" in response.json() assert agent.calls == [] - async def test_legacy_wire_format_is_explicit_once_warned_and_unchanged( - self, caplog: pytest.LogCaptureFixture - ) -> None: - with pytest.warns(DeprecationWarning, match="legacy_wire_format"): - server = InvocationsHostServer( - _make_agent(response_text="Hello!", stream_texts=["Hel", "lo", "!"]), legacy_wire_format=True - ) - with _request_context(session_id="legacy"): - for _ in range(2): - plain = await server._handle_invoke(_make_request({"message": "hi"})) # pyright: ignore[reportPrivateUsage] - assert bytes(plain.body).decode() == "Hello!" - streamed = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] - _make_request({"message": "hi", "stream": True}) - ) - assert isinstance(streamed, StreamingResponse) - assert await _collect_stream(streamed) == "Hello!" - - deprecations = [ - record - for record in caplog.records - if record.levelno == logging.WARNING and "DEPRECATION:" in record.message - ] - assert len(deprecations) == 1 - - async def test_legacy_stream_failure_terminates_instead_of_sending_new_frame(self) -> None: - class FailingAgent(_FakeAgent): - def run( - self, messages: Any = None, *, stream: bool = False, session: AgentSession | None = None, **kwargs: Any - ) -> Any: - async def updates() -> AsyncIterator[AgentResponseUpdate]: - yield AgentResponseUpdate(contents=[Content.from_text("first")]) - raise RuntimeError("provider failed") - - return updates() - - with pytest.warns(DeprecationWarning): - server = InvocationsHostServer(FailingAgent(), legacy_wire_format=True) - with _request_context(session_id="legacy-failure"): - response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] - _make_request({"message": "hi", "stream": True}) - ) - assert isinstance(response, StreamingResponse) - iterator = cast(Any, response.body_iterator) - assert await anext(iterator) == "first" - with pytest.raises(RuntimeError, match="provider failed"): - await anext(iterator) - # endregion @@ -1946,7 +1890,7 @@ async def test_reusing_session_restores_identifier(self, hosted: bool, stream: b with ( _request_context(call_id="call-1", session_id="sess-1", user_id="user-1"), patch( - "agent_framework_foundry_hosting._invocations_shared.AgentSession", wraps=AgentSession + "agent_framework_foundry_hosting._invocations_agent.AgentSession", wraps=AgentSession ) as session_factory, ): for _ in range(2): diff --git a/python/packages/foundry_hosting/tests/test_request.py b/python/packages/foundry_hosting/tests/test_request.py index 4a783d30556..740390d1eaa 100644 --- a/python/packages/foundry_hosting/tests/test_request.py +++ b/python/packages/foundry_hosting/tests/test_request.py @@ -18,7 +18,6 @@ response_run_options, validate_default_transport_options, validate_request_options, - validate_unsupported_options, ) @@ -56,19 +55,24 @@ def test_native_options_are_translated_before_flattened_extra_body() -> None: } -def test_nested_extra_body_is_also_overlaid_last_without_reserved_fields() -> None: +def test_nested_extra_body_is_overlaid_last_and_rejects_reserved_fields() -> None: request = cast( CreateResponse, { "max_output_tokens": 300, "extra_body": { "max_tokens": 150, - "session_id": "forged", - "continuation_token": {"response_id": "private"}, }, }, ) assert response_run_options(request) == {"max_tokens": 150} + with pytest.raises(ValueError, match="continuation_token, session_id"): + response_run_options( + cast( + CreateResponse, + {"extra_body": {"session_id": "forged", "continuation_token": {"response_id": "private"}}}, + ) + ) with pytest.raises(TypeError, match="extra_body must be a mapping"): response_run_options(cast(CreateResponse, {"extra_body": ["not a mapping"]})) with pytest.raises(ValueError, match="Nested extra_body is not supported"): @@ -148,16 +152,6 @@ def test_default_transport_cannot_override_host_storage_or_identity() -> None: validate_default_transport_options({"extra_body": {"temperature": 0.5}}, allow_agent_store=False) -@pytest.mark.parametrize("mode", ["ignore", "warn", "error"]) -def test_supported_unsupported_option_modes(mode: str) -> None: - assert validate_unsupported_options(mode) == mode - - -def test_unknown_unsupported_option_mode_fails_at_construction() -> None: - with pytest.raises(ValueError, match="unsupported_options"): - validate_unsupported_options("silent") - - def test_invocation_run_accepts_typed_messages_with_independent_defaults() -> None: first = InvocationRun(messages="hello") second = InvocationRun(messages="world", stream=True) diff --git a/python/packages/foundry_hosting/tests/test_responses.py b/python/packages/foundry_hosting/tests/test_responses.py index 04ae6bc1a38..e21fef917ea 100644 --- a/python/packages/foundry_hosting/tests/test_responses.py +++ b/python/packages/foundry_hosting/tests/test_responses.py @@ -17,7 +17,6 @@ import os import uuid from collections.abc import AsyncGenerator, AsyncIterator, Awaitable, Callable, Generator, Mapping, Sequence -from concurrent.futures import ThreadPoolExecutor from contextlib import suppress from dataclasses import dataclass from importlib import import_module @@ -71,6 +70,7 @@ from azure.ai.agentserver.responses.models import ( CreateResponse, Item, + MCPApprovalResponse, Metadata, OutputItem, ResponseIncompleteReason, @@ -85,10 +85,13 @@ from pydantic import TypeAdapter from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest +from agent_framework_foundry_hosting._approvals import SessionApprovalState from agent_framework_foundry_hosting._responses_shared import ( _INCOMPLETE_REASON_KEY, # pyright: ignore[reportPrivateUsage] _LATEST_CHECKPOINT_ID_KEY, # pyright: ignore[reportPrivateUsage] CONSENT_ERROR_CODE, + OAUTH_CONSENT_NOT_EXECUTED_RESULT, + OAUTH_CONSENT_REQUIRED_RESULT, ConsentError, _agent_response_updates, # pyright: ignore[reportPrivateUsage] _annotations_to_output_text, # pyright: ignore[reportPrivateUsage] @@ -109,7 +112,27 @@ from agent_framework_foundry_hosting._state_store import ( AgentSessionStoreProvider, FoundryAgentSessionStore, - FunctionApprovalStoreProvider, +) + +from .foundry_hosting.approval_helpers import SqliteSessionStore +from .foundry_hosting.responses_host_helpers import ( + ApprovalPublishingClient, + BackgroundAgentMiddleware, + BackgroundChatMiddleware, + BackgroundPollingClient, + FailingAgentMiddleware, + GatedAgentStream, + GatedRequestRead, + GatedSessionLoadStore, + StatefulConsentClient, + UnlayeredClient, + collect_host_events, +) +from .foundry_hosting.toolbox_consent_helpers import ( + CONSENT_LINK, + BatchToolCallClient, + ToolboxConsentServer, + consent_toolbox, ) _OPENAI_HTTPX = cast(Any, import_module(DefaultAsyncHttpxClient.__mro__[1].__module__.partition(".")[0])) @@ -1345,6 +1368,68 @@ def _sse_event_types(events: list[dict[str, Any]]) -> list[str]: # region Serialization Helpers +@pytest.mark.parametrize("stream", [False, True]) +async def test_agent_host_prefix_preserves_response_routes(stream: bool) -> None: + agent = _make_agent( + response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("Prefixed")])]) + ) + server = _make_server(agent, prefix="api/v1") + + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + created = await client.post("/api/v1/responses", json={"input": "hello", "stream": stream, "store": True}) + assert created.status_code == 200, created.text + if stream: + result = next( + event["data"]["response"] + for event in _parse_sse_events(created.text) + if event["event"] == "response.completed" + ) + else: + result = created.json() + assert result["status"] == "completed" + response_id = result["id"] + + stored = await client.get(f"/api/v1/responses/{response_id}") + assert stored.status_code == 200, stored.text + assert stored.json()["output"] == result["output"] + + inputs = await client.get(f"/api/v1/responses/{response_id}/input_items") + assert inputs.status_code == 200, inputs.text + assert inputs.json()["data"] + + deleted = await client.delete(f"/api/v1/responses/{response_id}") + assert deleted.status_code == 200, deleted.text + assert (await client.get(f"/api/v1/responses/{response_id}")).status_code == 404 + + +@pytest.mark.parametrize( + ("prefix", "path"), [("api/v1", "/api/v1"), (" /api/v1/ ", "/api/v1"), ("//api/v1/", "//api/v1")] +) +async def test_prefixed_agent_background_cancel_uses_sdk_routes(prefix: str, path: str) -> None: + gated = GatedAgentStream() + agent = _make_agent() + agent.run = MagicMock(side_effect=lambda **kwargs: ResponseStream(gated.updates())) + server = _make_server(agent, prefix=prefix) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + created = await client.post( + f"http://test{path}/responses", json={"input": "wait", "store": True, "background": True} + ) + assert created.status_code == 200, created.text + response_id = created.json()["id"] + await asyncio.wait_for(gated.started.wait(), 2) + cancelled = await client.post(f"http://test{path}/responses/{response_id}/cancel") + assert cancelled.status_code == 200, cancelled.text + await asyncio.wait_for(gated.stopped.wait(), 2) + for _ in range(150): + fetched = await client.get(f"http://test{path}/responses/{response_id}") + if fetched.status_code == 200 and fetched.json()["status"] == "cancelled": + break + await asyncio.sleep(0.01) + assert fetched.status_code == 200, fetched.text + assert fetched.json()["status"] == "cancelled" + assert "too late" not in fetched.text + + @pytest.mark.parametrize("stream", [False, True]) @pytest.mark.parametrize("metadata", [None, {}, {"foo": "bar", "view": "concise"}]) async def test_request_metadata_is_returned_and_stored(stream: bool, metadata: dict[str, str] | None) -> None: @@ -1580,6 +1665,67 @@ def test_stringify_mcp_output_extracts_only_text_content_mappings(self) -> None: class TestResponsesHostServerInit: + @pytest.mark.parametrize( + ("setting", "value", "error"), + [ + ("max_history_items", True, TypeError), + ("max_history_items", 0, ValueError), + ("sse_keep_alive_interval_seconds", False, TypeError), + ("sse_keep_alive_interval_seconds", 0, ValueError), + ("shutdown_grace_period_seconds", True, TypeError), + ("shutdown_grace_period_seconds", 0, ValueError), + ("enable_background_recovery", 1, TypeError), + ], + ) + def test_invalid_host_setting_fails_before_sdk_or_agent_mutation( + self, setting: str, value: Any, error: type[Exception] + ) -> None: + agent = _make_agent() + with ( + patch("azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost.__init__") as initialize, + pytest.raises(error, match=setting), + ): + ResponsesHostServer(agent, **{setting: value}) + initialize.assert_not_called() + assert agent.context_providers == [] + + def test_history_limit_is_an_explicit_host_managed_override(self) -> None: + default = _make_server(_make_agent()) + limited = _make_server(_make_agent(), max_history_items=3) + assert default._host_options.default_fetch_history_count == ( # pyright: ignore[reportPrivateUsage] + ResponsesServerOptions().default_fetch_history_count + ) + assert limited._host_options.default_fetch_history_count == 3 # pyright: ignore[reportPrivateUsage] + with pytest.warns(UserWarning, match="max_history_items is ignored"): + ignored = _make_server(_make_agent(), history_source="agent_managed", max_history_items=3) + assert ignored._host_options.default_fetch_history_count == ( # pyright: ignore[reportPrivateUsage] + ResponsesServerOptions().default_fetch_history_count + ) + + @pytest.mark.parametrize("env", [None, "0", "3"]) + @pytest.mark.parametrize("explicit", [None, 5]) + def test_heartbeat_precedence(self, monkeypatch: pytest.MonkeyPatch, env: str | None, explicit: int | None) -> None: + if env is None: + monkeypatch.delenv("SSE_KEEPALIVE_INTERVAL", raising=False) + else: + monkeypatch.setenv("SSE_KEEPALIVE_INTERVAL", env) + host = _make_server(_make_agent(), sse_keep_alive_interval_seconds=explicit) + assert host._endpoint._runtime_options.sse_keep_alive_interval_seconds == ( # pyright: ignore[reportPrivateUsage] + explicit if explicit is not None else (int(env) if env and int(env) > 0 else None) + ) + + def test_hidden_provider_background_fails_before_sdk_and_agent_mutation(self) -> None: + agent = _make_agent() + agent.default_options = {"background": True} + with ( + patch("azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost.__init__") as initialize, + pytest.raises(ValueError, match="outer background"), + ): + ResponsesHostServer(agent) + initialize.assert_not_called() + assert agent.default_options == {"background": True} + assert agent.context_providers == [] + def test_rejects_workflow_agent_before_sdk_construction(self) -> None: agent = _WorkflowAgentMock() with ( @@ -1622,10 +1768,7 @@ def test_init_uses_default_store_providers(self) -> None: server._session_storage_provider, # pyright: ignore[reportPrivateUsage] AgentSessionStoreProvider, ) - assert isinstance( - server._function_approval_storage_provider, # pyright: ignore[reportPrivateUsage] - FunctionApprovalStoreProvider, - ) + assert not hasattr(server, "_function_approval_storage_provider") def test_init_rejects_history_provider_with_load_messages(self) -> None: @@ -1708,10 +1851,10 @@ def test_failed_init_does_not_mutate_agent(self) -> None: default_options={"store": True}, ) - with pytest.raises(RuntimeError, match="resilient_background"): + with pytest.raises(ValueError, match="enable_background_recovery"): ResponsesHostServer( agent, - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, ) assert agent.default_options["store"] is True @@ -1721,50 +1864,18 @@ def test_init_rejects_resilient_background_for_non_workflow_agent(self, tmp_path agent = _make_agent( response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("hi")])]) ) - with pytest.raises(RuntimeError, match="resilient_background"): + with pytest.raises(ValueError, match="enable_background_recovery"): ResponsesHostServer( agent, response_store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, ) - def test_init_rejects_steerable_conversations(self) -> None: - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable"): - ResponsesHostServer( - _make_agent(), - response_store=InMemoryResponseProvider(), - options=ResponsesServerOptions(steerable_conversations=True), - ) - - def test_steering_rejected_before_enabling_task_manager_or_starting_host(self) -> None: - agent_factory = MagicMock() - - def construct(_turn: int) -> str: - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable") as error: - ResponsesHostServer( - agent=agent_factory, - options=ResponsesServerOptions(steerable_conversations=True), - ) - return str(error.value) - - with ( - patch("azure.ai.agentserver.core.tasks.set_resilient_tasks_enabled") as enable, - patch("agent_framework_foundry_hosting._responses_agent.ResponsesAgentServerHost.__init__") as base_init, - ThreadPoolExecutor(max_workers=12) as executor, - ): - failures = list(executor.map(construct, range(24))) - - assert len(failures) == 24 - assert all("Azure/azure-sdk-for-python#49233" in failure for failure in failures) - enable.assert_not_called() - base_init.assert_not_called() - agent_factory.assert_not_called() - async def test_non_steerable_background_still_uses_outer_response_id(self) -> None: agent = _make_agent( stream_updates=[AgentResponseUpdate(contents=[Content.from_text("finished")], role="assistant")] ) - server = _make_server(agent, options=ResponsesServerOptions(steerable_conversations=False)) + server = _make_server(agent) async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as http: pending = await http.post("/responses", json={"input": "hello", "store": True, "background": True}) @@ -1781,32 +1892,10 @@ async def test_non_steerable_background_still_uses_outer_response_id(self) -> No assert final.json()["id"] == response_id assert "finished" in str(final.json()["output"]) - def test_provider_background_rejects_steering_and_wrong_history_source(self) -> None: + def test_provider_background_rejects_wrong_history_source(self) -> None: agent = Agent(client=_ServiceStorageRecordingClient()) with pytest.raises(ValueError, match="history_source='inference_service'"): _make_server(agent, background_source="inference_service") - with pytest.raises(RuntimeError, match="temporarily unavailable"): - _make_server( - agent, - history_source="inference_service", - background_source="inference_service", - options=ResponsesServerOptions(steerable_conversations=True), - ) - - @pytest.mark.parametrize("old_store", [None, InMemoryResponseProvider()]) - @pytest.mark.parametrize("with_response_store", [False, True]) - def test_removed_store_alias_is_rejected_before_sdk_construction( - self, old_store: Any, with_response_store: bool - ) -> None: - agent = _make_agent() - kwargs: dict[str, Any] = {"store": old_store} - if with_response_store: - kwargs["response_store"] = InMemoryResponseProvider() - with patch("azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost.__init__") as initialize: - with pytest.raises(TypeError, match="store= alias was removed; use response_store="): - ResponsesHostServer(agent, **kwargs) - initialize.assert_not_called() - assert agent.context_providers == [] def test_history_and_background_sources_reject_invalid_values(self) -> None: agent = _make_agent() @@ -1814,8 +1903,6 @@ def test_history_and_background_sources_reject_invalid_values(self) -> None: _make_server(agent, history_source="host") with pytest.raises(ValueError, match="background_source"): _make_server(agent, background_source="host") - with pytest.raises(ValueError, match="unsupported_options"): - _make_server(agent, unsupported_options="silent") @pytest.mark.parametrize( ("setting", "value"), @@ -1827,7 +1914,7 @@ def test_history_and_background_sources_reject_invalid_values(self) -> None: ("background_source", "provider"), ], ) - def test_legacy_source_literals_are_rejected(self, setting: str, value: str) -> None: + def test_unknown_source_literals_are_rejected(self, setting: str, value: str) -> None: with pytest.raises(ValueError, match=setting): _make_server(_make_agent(), **{setting: value}) @@ -1864,6 +1951,157 @@ async def test_previous_response_requires_existing_agent_session(self) -> None: agent.run.assert_not_called() agent.create_session.assert_not_called() + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("kind", ["agent", "chat"]) + async def test_application_middleware_background_is_not_intercepted(self, stream: bool, kind: str) -> None: + client = StatefulConsentClient(["tool"]) + middleware = ( + BackgroundAgentMiddleware(nested=False) if kind == "agent" else BackgroundChatMiddleware(nested=False) + ) + agent = Agent(client=client, middleware=[middleware]) + server = _make_server(agent, history_source="inference_service") + + response = await _post(server, stream=stream) + + assert response.status_code == 200 + if stream: + assert "response.completed" in _sse_event_types(_parse_sse_events(response.text)) + else: + assert response.json()["status"] == "completed" + assert client.calls + assert client.options[0]["background"] is True + assert agent.middleware == [middleware] + + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("background_source", ["host_managed", "inference_service"]) + @pytest.mark.parametrize("hook_background", [None, False, True]) + async def test_hook_background_is_forwarded_on_ordinary_runs( + self, stream: bool, background_source: str, hook_background: bool | None + ) -> None: + client = StatefulConsentClient(["tool"]) + server = _make_server( + Agent(client=client), + history_source="inference_service", + background_source=background_source, + prepare_options=( + (lambda _request, _options: {"background": hook_background}) if hook_background is not None else None + ), + ) + + response = await _post(server, stream=stream) + + assert response.status_code == 200 + if stream: + assert "response.completed" in _sse_event_types(_parse_sse_events(response.text)) + else: + assert response.json()["status"] == "completed" + assert len(client.calls) == 1 + if hook_background is not None: + assert client.options[0]["background"] is hook_background + elif background_source == "inference_service": + assert client.options[0]["background"] is False + else: + assert "background" not in client.options[0] + + @pytest.mark.parametrize("conflicting_hook", [False, True]) + async def test_host_selected_provider_background_retains_submit_and_poll_precedence( + self, conflicting_hook: bool + ) -> None: + client = BackgroundPollingClient() + agent = Agent(client=client) + server = _make_server( + agent, + session_store=SessionStore(), + history_source="inference_service", + background_source="inference_service", + prepare_options=(lambda _request, _options: {"background": False}) if conflicting_hook else None, + ) + request = CreateResponse(input="hello", store=True, background=True) + context = ResponseContext(response_id="outer-response", mode_flags=MagicMock()) + with ( + patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), + ): + events = [event async for event in server._handle_response(request, context, asyncio.Event())] + + assert any(isinstance(event, Mapping) and event.get("type") == "response.completed" for event in events) + assert [call["background"] for call in client.calls] == [True, True] + assert client.calls[1]["continuation_token"] == {"response_id": "provider-token"} + assert not agent.middleware + assert not client.chat_middleware + + async def test_raw_agent_client_chat_middleware_background_is_not_intercepted(self) -> None: + client = StatefulConsentClient(["tool"]) + middleware = BackgroundChatMiddleware(nested=False) + client.chat_middleware.append(middleware) + agent = RawAgent(client=client) + server = _make_server( + agent, + session_store=SessionStore(), + history_source="inference_service", + background_source="inference_service", + ) + request = CreateResponse(input="hello", store=True, background=False) + context = ResponseContext(response_id="outer-response", mode_flags=MagicMock()) + with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])): + events = [event async for event in server._handle_response(request, context, asyncio.Event())] + + assert any(isinstance(event, Mapping) and event.get("type") == "response.completed" for event in events) + assert len(client.calls) == 1 + assert client.options[0]["background"] is True + assert client.chat_middleware == [middleware] + + @pytest.mark.parametrize("stream", [False, True]) + async def test_middleware_failure_on_approval_reply_leaves_parent_pending( + self, stream: bool, tmp_path: Path + ) -> None: + client = ApprovalPublishingClient(_make_function_approval_request_content()) + agent = Agent(client=client, middleware=[]) + server = _make_server(agent, session_store=SqliteSessionStore(tmp_path / "middleware-approval.sqlite")) + first = await _post(server) + approval_id = next(item["id"] for item in first.json()["output"] if item["type"] == "mcp_approval_request") + agent.middleware = [FailingAgentMiddleware()] + payload = { + "input": [{"type": "mcp_approval_response", "approval_request_id": approval_id, "approve": True}], + "previous_response_id": first.json()["id"], + "stream": stream, + } + + rejected = await _post_json(server, payload) + + assert rejected.status_code == 200 + if stream: + assert "response.failed" in _sse_event_types(_parse_sse_events(rejected.text)) + else: + assert rejected.json()["status"] == "failed" + assert client.calls == 1 + # The failed turn saved nothing, so the parent still holds the pending request. + agent.middleware = [] + replay = await _post_json(server, payload) + assert replay.status_code == 200 + assert client.calls == 2 + + @pytest.mark.parametrize("provider_background", [False, True]) + @pytest.mark.parametrize("layered_agent", [False, True]) + async def test_unlayered_clients_receive_no_middleware_kwarg( + self, provider_background: bool, layered_agent: bool + ) -> None: + client = UnlayeredClient() + agent = Agent(client=client) if layered_agent else RawAgent(client=client) + server = _make_server( + agent, + history_source="inference_service", + background_source="inference_service" if provider_background else "host_managed", + ) + request = CreateResponse(input="hello", store=True, background=provider_background) + context = ResponseContext(response_id="unlayered-response", mode_flags=MagicMock()) + with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])): + events = [event async for event in server._handle_response(request, context, asyncio.Event())] + + assert any(isinstance(event, Mapping) and event.get("type") == "response.completed" for event in events) + assert client.kwargs + assert all("middleware" not in kwargs for kwargs in client.kwargs) + # endregion @@ -2160,8 +2398,9 @@ async def test_invalid_approval_does_not_consume_service_parent(self) -> None: }, ) + assert invalid.status_code == 200 assert invalid.json()["status"] == "failed" - assert "unknown-approval" in invalid.json()["error"]["message"] + assert "No approval reply matches" in invalid.json()["error"]["message"] parent = await store.get(parent_id) assert parent is not None and "_foundry_service_child" not in parent.state assert len(client.calls) == 1 @@ -2201,8 +2440,6 @@ async def test_store_false_neither_saves_session_nor_stores_inner_response(self, agent = Agent(client=client, default_options=OpenAIChatOptions(store=True)) store = SessionStore() server = _make_server(agent, session_store=store, history_source=mode) - approvals = MagicMock(spec=FunctionApprovalStoreProvider) - server._function_approval_storage_provider = approvals # pyright: ignore[reportPrivateUsage] response = await _post_json(server, {"input": "one shot", "store": False, "model": "test-model"}) @@ -2213,7 +2450,6 @@ async def test_store_false_neither_saves_session_nor_stores_inner_response(self, assert agent.default_options["store"] is True provider = cast(MagicMock, server._session_storage_provider) # pyright: ignore[reportPrivateUsage] provider.get_store.assert_not_called() - approvals.get_store.assert_not_called() async def test_store_false_rejects_external_history_and_custom_agent(self) -> None: history = _PerServiceCallHistoryProvider() @@ -2245,6 +2481,35 @@ async def test_store_false_agent_continuation_fails_instead_of_using_service_his assert "store=false cannot continue agent-managed downstream service history" in _failure_message(events) assert len(client.calls) == 1 + @pytest.mark.parametrize("stream", [False, True]) + async def test_caller_options_without_hook_preserve_agent_defaults(self, stream: bool) -> None: + client = _RecordingHistoryClient() + agent = Agent(client=client, default_options=OpenAIChatOptions(temperature=0.2, max_tokens=256)) + defaults = dict(agent.default_options) + server = _make_server(agent) + + response = await _post_json( + server, + { + "input": "hello", + "stream": stream, + "temperature": 0.8, + "max_output_tokens": 300, + "extra_body": {"top_p": 0.4}, + }, + ) + + assert response.status_code == 200 + if stream: + assert _sse_event_types(_parse_sse_events(response.text))[-1] == "response.completed" + else: + assert response.json()["status"] == "completed" + assert len(client.options) == 1 + assert client.options[0]["temperature"] == 0.2 + assert client.options[0]["max_tokens"] == 256 + assert "top_p" not in client.options[0] + assert agent.default_options == defaults + async def test_extra_options_overlay_then_developer_hook_preserves_defaults(self) -> None: client = _RecordingHistoryClient() agent = Agent(client=client, default_options=OpenAIChatOptions(temperature=0.2, max_tokens=256)) @@ -2339,7 +2604,7 @@ def handle_request(request: Any) -> Any: max_retries=0, ) as openai: agent = Agent(client=OpenAIChatClient(model="test-model", async_client=openai)) - server = _make_server(agent) + server = _make_server(agent, prepare_options=lambda _request, options: options) rejected = await _post_json( server, {"input": "unsafe", "store": False, "extra_body": {"extra_body": {"store": True}}}, @@ -2348,12 +2613,12 @@ def handle_request(request: Any) -> Any: assert "Nested extra_body" in rejected.json()["error"]["message"] assert outbound == [] - legacy_default = Agent( + default_store_agent = Agent( client=OpenAIChatClient(model="test-model", async_client=openai), default_options=cast(Any, {"extra_body": {"store": True}}), ) - legacy_server = _make_server(legacy_default, history_source="agent_managed") - rejected_default = await _post_json(legacy_server, {"input": "unsafe", "store": False}) + default_store_server = _make_server(default_store_agent, history_source="agent_managed") + rejected_default = await _post_json(default_store_server, {"input": "unsafe", "store": False}) assert rejected_default.json()["status"] == "failed" assert "default extra_body" in rejected_default.json()["error"]["message"] assert outbound == [] @@ -2369,27 +2634,27 @@ def handle_request(request: Any) -> Any: assert outbound[0]["temperature"] == 0.1 assert "extra_body" not in outbound[0] - async def test_unsupported_options_policy_rejects_custom_agent(self) -> None: + async def test_custom_agent_runs_messages_only_without_prepare_options(self) -> None: agent = _StrictCustomAgent() - server = _make_server(agent, history_source="agent_managed", unsupported_options="error") - response = await _post_json(server, {"input": "hello", "temperature": 0.8}) + server = _make_server(agent, history_source="agent_managed") - assert response.json()["status"] == "failed" - assert "does not accept caller runtime options" in response.json()["error"]["message"] - assert agent.calls == [] + response = await _post(server, temperature=0.5) - @pytest.mark.parametrize(("policy", "warn"), [("warn", True), ("ignore", False)]) - async def test_custom_agent_unsupported_options_warning_policy( - self, policy: str, warn: bool, caplog: pytest.LogCaptureFixture - ) -> None: - agent = _StrictCustomAgent() - server = _make_server(agent, history_source="agent_managed", unsupported_options=policy) + assert response.json()["status"] == "completed" + assert len(agent.calls) == 1 - with caplog.at_level(logging.WARNING): - response = await _post(server, temperature=0.5) + async def test_prepare_options_fails_for_agent_without_options_parameter(self) -> None: + agent = _StrictCustomAgent() + server = _make_server( + agent, + history_source="agent_managed", + prepare_options=lambda _request, options: {**options, "temperature": 0.8}, + ) + response = await _post_json(server, {"input": "hello"}) - assert response.json()["status"] == "completed" - assert ("Agent doesn't support runtime options" in caplog.text) is warn + assert response.json()["status"] == "failed" + assert "does not accept the run options returned by prepare_options" in response.json()["error"]["message"] + assert agent.calls == [] @pytest.mark.parametrize( ("finish_reason", "expected_status"), @@ -2495,7 +2760,7 @@ async def run(messages: Any = None, *, session: AgentSession, **kwargs: Any) -> session_store=SessionStore(), history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -2555,8 +2820,9 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), + prepare_options=lambda _request, options: options, ) request = CreateResponse(input="hello", store=True, background=True, temperature=0.35) context = ResponseContext(response_id="outer-response", mode_flags=MagicMock()) @@ -2629,7 +2895,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) context = ResponseContext( @@ -2665,7 +2931,7 @@ async def test_provider_recovery_without_saved_token_does_not_restart_job( session_store=SessionStore(), history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -2708,7 +2974,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse( @@ -2740,7 +3006,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) recovered_context = ResponseContext(response_id="outer-recover", mode_flags=MagicMock()) @@ -2790,8 +3056,9 @@ def make_host() -> ResponsesHostServer: agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), + prepare_options=lambda _request, options: options, ) request = CreateResponse(input="hello", store=True, background=True, temperature=0.25) @@ -2876,7 +3143,7 @@ def make_host() -> ResponsesHostServer: agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3066,8 +3333,9 @@ async def set(self, session_id: str, session: AgentSession) -> None: session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), + prepare_options=lambda _request, options: options, ) context = ResponseContext(response_id="outer-tool-response", mode_flags=MagicMock()) request = CreateResponse(input="email bob", store=True, background=True, temperature=0.42) @@ -3209,7 +3477,7 @@ async def sleep(_seconds: float) -> None: session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) response_id = f"outer-{stage}-{interruption}" @@ -3325,7 +3593,7 @@ def make_host() -> ResponsesHostServer: agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3434,7 +3702,7 @@ def make_host() -> ResponsesHostServer: session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3513,7 +3781,7 @@ async def run( session_store=FailingTokenStore() if phase == "save" else SessionStore(), history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -3534,63 +3802,6 @@ async def run( exception = record.exc_info[1] assert exception is not None and exception.__cause__ is None and exception.__context__ is None - async def test_steering_keeps_superseded_snapshot_without_replacing_conversation_head(self) -> None: - store = SessionStore() - await store.set("conversation-head", AgentSession()) - first_continues = asyncio.Event() - calls = 0 - agent = _make_agent() - - def run_with_state(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate, AgentResponse]: - nonlocal calls - del args - calls += 1 - turn = calls - kwargs["session"].state["turn"] = turn - - async def updates() -> AsyncIterator[AgentResponseUpdate]: - yield AgentResponseUpdate(contents=[Content.from_text(f"turn {turn}")], role="assistant") - if turn == 1: - await first_continues.wait() - yield AgentResponseUpdate(contents=[Content.from_text("stale")], role="assistant") - - return ResponseStream(updates(), finalizer=AgentResponse.from_updates) - - agent.run = MagicMock(side_effect=run_with_state) - server = _make_server(agent, session_store=store) - # Exercise the handler's superseded-turn snapshot invariant without starting SDK steering. - server._host_options = ResponsesServerOptions(steerable_conversations=True) # pyright: ignore[reportPrivateUsage] - first = ResponseContext( - response_id="first-response", conversation_id="conversation-head", mode_flags=MagicMock() - ) - second = ResponseContext( - response_id="second-response", conversation_id="conversation-head", mode_flags=MagicMock() - ) - first_signal = asyncio.Event() - with ( - patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), - ): - first_handler = server._handle_response(CreateResponse(input="first"), first, first_signal) - async for event in first_handler: - if isinstance(event, Mapping) and event.get("type") == "response.output_text.delta": - break - newer = [ - event - async for event in server._handle_response(CreateResponse(input="second"), second, asyncio.Event()) - ] - first_signal.set() - older = [event async for event in first_handler] - - assert isinstance(newer[-1], Mapping) and newer[-1]["type"] == "response.completed" - assert isinstance(older[-1], Mapping) and older[-1]["type"] == "response.completed" - first_snapshot = await store.get("first-response") - second_snapshot = await store.get("second-response") - conversation_head = await store.get("conversation-head") - assert first_snapshot is not None and first_snapshot.state["turn"] == 1 - assert second_snapshot is not None and second_snapshot.state["turn"] == 2 - assert conversation_head is not None and conversation_head.state["turn"] == 2 - @pytest.mark.parametrize("existing_head", [False, True]) async def test_service_conversation_claim_prevents_parallel_provider_dispatch(self, existing_head: bool) -> None: conversation_id = f"conversation-{uuid.uuid4().hex}" @@ -5134,7 +5345,7 @@ async def test_chat_options_forwarded(self) -> None: response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]), raw_agent=True, ) - server = _make_server(agent) + server = _make_server(agent, prepare_options=lambda _request, options: options) resp = await _post( server, stream=False, @@ -5176,7 +5387,7 @@ async def test_chat_options_forwarded(self) -> None: stream_updates=[AgentResponseUpdate(contents=[Content.from_text("ok")], role="assistant")], raw_agent=True, ) - server = _make_server(agent) + server = _make_server(agent, prepare_options=lambda _request, options: options) resp = await _post( server, stream=True, @@ -6696,7 +6907,6 @@ async def test_mcp_approval_request(self) -> None: assert msg.contents[0].type == "function_approval_request" async def test_mcp_approval_response(self) -> None: - from azure.ai.agentserver.responses.models import MCPApprovalResponse saved = _make_function_approval_request_content(request_id="apr-1") storage = _function_approval_store(saved) @@ -8121,7 +8331,6 @@ async def test_input_item_mcp_approval_request_loads_from_storage(self) -> None: assert msg.contents[0].id == "apr-1" async def test_input_item_mcp_approval_response_resolves_to_approval_response(self) -> None: - from azure.ai.agentserver.responses.models import MCPApprovalResponse saved = _make_function_approval_request_content(request_id="apr-1") storage = _function_approval_store(saved) @@ -8150,10 +8359,11 @@ class TestFunctionApprovalRoundTrip: `function_approval_response` content delivered to the agent. """ - async def test_non_streaming_emits_mcp_approval_request_and_persists_to_storage(self) -> None: + async def test_non_streaming_emits_mcp_approval_request_and_persists_to_storage(self, tmp_path: Path) -> None: request_content = _make_function_approval_request_content() agent = _make_agent(response=AgentResponse(messages=[Message(role="assistant", contents=[request_content])])) - server = _make_server(agent) + store = SqliteSessionStore(tmp_path / "approvals.sqlite") + server = _make_server(agent, session_store=store) resp = await _post(server, stream=False) @@ -8167,15 +8377,17 @@ async def test_non_streaming_emits_mcp_approval_request_and_persists_to_storage( assert approval_items[0]["server_label"] == "my_server" # Storage must contain a saved entry under the emitted request id. - loaded = await server._function_approval_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] - config=server.config, platform_context=get_request_context() - ).load_approval_request(approval_request_id) + session = await store.get(body["id"]) + assert session is not None + loaded = await SessionApprovalState(session).load_approval_request(approval_request_id) assert loaded.type == "function_approval_request" assert loaded.function_call is not None assert loaded.function_call.name == "delete_file" @pytest.mark.parametrize("with_preamble", [False, True]) - async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self, with_preamble: bool) -> None: + async def test_streaming_emits_mcp_approval_request_and_persists_to_storage( + self, with_preamble: bool, tmp_path: Path + ) -> None: request_content = _make_function_approval_request_content(request_id="apr_streaming") updates: list[AgentResponseUpdate] = [] preambles = ("Checking the refund policy.", "The refund requires approval.") @@ -8202,7 +8414,8 @@ async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self ) updates.append(AgentResponseUpdate(contents=[request_content], role="assistant")) agent = _make_agent(stream_updates=updates) - server = _make_server(agent) + store = SqliteSessionStore(tmp_path / "approvals.sqlite") + server = _make_server(agent, session_store=store) resp = await _post(server, stream=True) @@ -8227,12 +8440,13 @@ async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self break assert approval_request_id is not None - loaded = await server._function_approval_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] - config=server.config, platform_context=get_request_context() - ).load_approval_request(approval_request_id) + response_id = next(e["data"]["response"]["id"] for e in events if e["event"] == "response.completed") + session = await store.get(response_id) + assert session is not None + loaded = await SessionApprovalState(session).load_approval_request(approval_request_id) assert loaded.type == "function_approval_request" - async def test_round_trip_approval_response_reaches_agent(self) -> None: + async def test_round_trip_approval_response_reaches_agent(self, tmp_path: Path) -> None: """Two-turn: turn 1 emits an approval request; turn 2 sends an approval response and the agent receives a `function_approval_response`.""" request_content = _make_function_approval_request_content() @@ -8243,7 +8457,7 @@ async def test_round_trip_approval_response_reaches_agent(self) -> None: AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("done")])]), ] ) - server = _make_server(agent) + server = _make_server(agent, session_store=SqliteSessionStore(tmp_path / "approvals.sqlite")) first = await _post(server, stream=False) assert first.status_code == 200 @@ -8263,6 +8477,7 @@ async def test_round_trip_approval_response_reaches_agent(self) -> None: } ], "stream": False, + "previous_response_id": first_body["id"], } second = await _post_json(server, second_payload) assert second.status_code == 200 @@ -8278,7 +8493,7 @@ async def test_round_trip_approval_response_reaches_agent(self) -> None: assert approval_responses[0].approved is True assert approval_responses[0].function_call.name == "delete_file" - async def test_round_trip_approval_response_rejected(self) -> None: + async def test_round_trip_approval_response_rejected(self, tmp_path: Path) -> None: """Same as above but the user rejects the approval; the agent must receive `approved=False`.""" request_content = _make_function_approval_request_content() @@ -8289,7 +8504,7 @@ async def test_round_trip_approval_response_rejected(self) -> None: AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]), ] ) - server = _make_server(agent) + server = _make_server(agent, session_store=SqliteSessionStore(tmp_path / "approvals.sqlite")) first = await _post(server, stream=False) approval_request_id = next( @@ -8308,6 +8523,7 @@ async def test_round_trip_approval_response_rejected(self) -> None: } ], "stream": False, + "previous_response_id": first.json()["id"], }, ) assert second.status_code == 200 @@ -8319,12 +8535,37 @@ async def test_round_trip_approval_response_rejected(self) -> None: assert len(approval_responses) == 1 assert approval_responses[0].approved is False - async def test_approval_response_referencing_unknown_id_fails(self) -> None: - """Sending an `mcp_approval_response` for a request id that was - never persisted must surface as a ``response.failed`` event whose - ``error.message`` contains the missing approval request id.""" + @pytest.mark.parametrize("stream", [False, True]) + async def test_approval_response_referencing_unknown_id_fails_without_running(self, stream: bool) -> None: + """An unknown wire ID is not translated; a reply with nothing left fails before the agent runs.""" agent = _make_agent( - response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]) + response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]), + stream_updates=[AgentResponseUpdate(contents=[Content.from_text("ok")], role="assistant")], + ) + server = _make_server(agent) + + resp = await _post_json( + server, + { + "model": "test-model", + "input": [{"type": "mcp_approval_response", "approval_request_id": "apr_unknown", "approve": True}], + "stream": stream, + }, + ) + + assert resp.status_code == 200, resp.text + if stream: + assert "response.failed" in _sse_event_types(_parse_sse_events(resp.text)) + else: + assert resp.json()["status"] == "failed" + agent.run.assert_not_called() + + @pytest.mark.parametrize("stream", [False, True]) + async def test_unknown_approval_id_is_dropped_beside_other_input(self, stream: bool) -> None: + """An unknown wire ID beside other input is dropped; Core never sees an approval to authorize.""" + agent = _make_agent( + response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]), + stream_updates=[AgentResponseUpdate(contents=[Content.from_text("ok")], role="assistant")], ) server = _make_server(agent) @@ -8333,24 +8574,53 @@ async def test_approval_response_referencing_unknown_id_fails(self) -> None: { "model": "test-model", "input": [ - { - "type": "mcp_approval_response", - "approval_request_id": "apr_unknown", - "approve": True, - } + {"type": "mcp_approval_response", "approval_request_id": "apr_unknown", "approve": True}, + {"type": "message", "role": "user", "content": "hello"}, ], - "stream": False, + "stream": stream, }, ) - # The handler converts the underlying KeyError into a terminal - # ``response.failed`` event, so non-streaming callers see HTTP 200 - # with status="failed" and a meaningful error message rather than - # a generic 5xx response. - assert resp.status_code == 200 - body = resp.json() - assert body["status"] == "failed" - error: dict[str, Any] = body.get("error") or {} - assert "apr_unknown" in (error.get("message") or "") + + assert resp.status_code == 200, resp.text + assert agent.run.call_count == 1 + sent = [content for message in agent.run.call_args.kwargs["messages"] for content in message.contents] + assert not any(content.type.startswith("function_approval") for content in sent) + assert any(content.type == "text" and content.text == "hello" for content in sent) + + @pytest.mark.parametrize("stream", [False, True]) + async def test_approval_from_another_platform_user_is_dropped(self, stream: bool, tmp_path: Path) -> None: + approval = _make_function_approval_request_content() + agent = _make_multi_response_agent([ + AgentResponse(messages=[Message(role="assistant", contents=[approval])]), + AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("other")])]), + AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("done")])]), + ]) + stores = { + "owner": SqliteSessionStore(tmp_path / "owner.sqlite"), + "other": SqliteSessionStore(tmp_path / "other.sqlite"), + } + provider = MagicMock(spec=AgentSessionStoreProvider) + provider.get_store.side_effect = lambda **kwargs: stores[kwargs["platform_context"].user_id] + server = _make_server(agent, agent_session_store_provider=provider) + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + first = await client.post("/responses", json={"input": "go"}, headers={"x-agent-user-id": "owner"}) + assert first.status_code == 200, first.text + approval_id = next(item["id"] for item in first.json()["output"] if item["type"] == "mcp_approval_request") + payload = { + "input": [{"type": "mcp_approval_response", "approval_request_id": approval_id, "approve": True}], + "previous_response_id": first.json()["id"], + "stream": stream, + } + await client.post("/responses", json=payload, headers={"x-agent-user-id": "other"}) + if agent.run.call_count == 2: + sent = [c for m in agent.run.call_args.kwargs["messages"] for c in m.contents] + assert not any(content.type == "function_approval_response" for content in sent) + calls_before_owner = agent.run.call_count + accepted = await client.post("/responses", json=payload, headers={"x-agent-user-id": "owner"}) + assert accepted.status_code == 200, accepted.text + assert agent.run.call_count == calls_before_owner + 1 + sent = [c for m in agent.run.call_args.kwargs["messages"] for c in m.contents] + assert [content.approved for content in sent if content.type == "function_approval_response"] == [True] # endregion @@ -8361,6 +8631,178 @@ async def test_approval_response_referencing_unknown_id_fails(self) -> None: # region Agent lifecycle (lazy entry & OAuth consent surfacing) +@pytest.mark.xdist_group("session_isolation") +class TestToolboxConsentHost: + @pytest.mark.parametrize("stream", [False, True]) + async def test_consent_preserves_function_results_and_stored_continuation( + self, stream: bool, tmp_path: Path + ) -> None: + gateway = ToolboxConsentServer(["before", "secure", "after"]) + gateway.require_consent("secure") + async with consent_toolbox(gateway) as toolbox: + client = BatchToolCallClient([function.name for function in toolbox.functions]) + client.function_invocation_configuration["allow_concurrent_invocation"] = False # type: ignore[index] + agent = Agent(client=client, tools=toolbox, default_options=ChatOptions(allow_multiple_tool_calls=True)) + server = _make_server(agent, session_store=SqliteSessionStore(tmp_path / "consent.sqlite")) + + first = await _post(server, stream=stream) + assert first.status_code == 200, first.text + if stream: + events = _parse_sse_events(first.text) + response = next( + event["data"]["response"] for event in events if event["event"] == "response.incomplete" + ) + else: + response = first.json() + assert response["status"] == "incomplete" + outputs = response["output"] + calls = [item for item in outputs if item["type"] == "function_call"] + results = [item for item in outputs if item["type"] == "function_call_output"] + consents = [item for item in outputs if item["type"] == "oauth_consent_request"] + assert len(calls) == len(results) == 3 + assert {item["call_id"] for item in calls} == {item["call_id"] for item in results} + assert len({item["call_id"] for item in results}) == 3 + by_call = {item["call_id"]: item["output"] for item in results} + assert "before-result" in str(by_call["call-0"]) + assert by_call["call-1"] == OAUTH_CONSENT_REQUIRED_RESULT + assert by_call["call-2"] == OAUTH_CONSENT_NOT_EXECUTED_RESULT + assert len(consents) == 1 + assert consents[0]["consent_link"] == CONSENT_LINK + assert gateway.calls == ["before", "secure"] + + gateway.consent_errors.clear() + second = await _post_json( + server, + {"input": "Consent granted", "previous_response_id": response["id"], "stream": stream}, + ) + assert second.status_code == 200, second.text + if stream: + assert "response.completed" in _sse_event_types(_parse_sse_events(second.text)) + else: + assert second.json()["status"] == "completed" + assert len(client.calls) == 2 + history_results = [ + content.call_id + for message in client.calls[1] + for content in message.contents + if content.type == "function_result" + ] + assert len(history_results) == 3 + assert set(history_results) == {"call-0", "call-1", "call-2"} + + @pytest.mark.parametrize("stream", [False, True]) + async def test_service_history_fails_on_invocation_time_consent_before_publication( + self, stream: bool, tmp_path: Path + ) -> None: + gateway = ToolboxConsentServer(["before", "secure", "after"]) + gateway.require_consent("secure") + async with consent_toolbox(gateway) as toolbox: + client = StatefulConsentClient([function.name for function in toolbox.functions]) + client.function_invocation_configuration["allow_concurrent_invocation"] = False # type: ignore[index] + agent = Agent(client=client, tools=toolbox, default_options=ChatOptions(allow_multiple_tool_calls=True)) + server = _make_server( + agent, + session_store=SqliteSessionStore(tmp_path / "service-consent.sqlite"), + history_source="inference_service", + ) + response = await _post(server, stream=stream) + assert response.status_code == 200, response.text + assert CONSENT_LINK not in response.text + assert "oauth_consent_request" not in response.text + assert ( + "Invocation-time OAuth consent is unsupported with history_source='inference_service'." in response.text + ) + if stream: + events = _parse_sse_events(response.text) + assert "response.failed" in _sse_event_types(events) + assert "response.incomplete" not in _sse_event_types(events) + else: + assert response.json()["status"] == "failed" + assert len(client.calls) == 2 + assert gateway.calls == ["before", "before", "secure"] + + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("background_source", ["host_managed", "inference_service"]) + async def test_service_history_background_fails_on_invocation_time_consent_before_publication( + self, stream: bool, background_source: str, tmp_path: Path + ) -> None: + gateway = ToolboxConsentServer(["before", "secure", "after"]) + gateway.require_consent("secure") + async with consent_toolbox(gateway) as toolbox: + client = StatefulConsentClient([function.name for function in toolbox.functions]) + client.function_invocation_configuration["allow_concurrent_invocation"] = False # type: ignore[index] + agent = Agent(client=client, tools=toolbox) + server = _make_server( + agent, + session_store=SqliteSessionStore(tmp_path / "background-consent.sqlite"), + history_source="inference_service", + background_source=background_source, + ) + request = CreateResponse(input="go", store=True, background=True, stream=stream) + context = ResponseContext(response_id="background-consent", mode_flags=MagicMock()) + with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])): + events = [event async for event in server._handle_response(request, context, asyncio.Event())] + + assert CONSENT_LINK not in str(events) + assert "oauth_consent_request" not in str(events) + assert "Invocation-time OAuth consent is unsupported with history_source='inference_service'." in str(events) + types = [event.get("type") for event in events if isinstance(event, Mapping)] + assert "response.failed" in types + assert "response.incomplete" not in types + assert gateway.calls == ["before", "before", "secure"] + + @pytest.mark.parametrize("stream", [False, True]) + async def test_new_store_false_consent_fails_without_exposing_continuation( + self, stream: bool, tmp_path: Path + ) -> None: + gateway = ToolboxConsentServer(["secure"]) + gateway.require_consent("secure") + async with consent_toolbox(gateway) as toolbox: + client = BatchToolCallClient([function.name for function in toolbox.functions]) + agent = Agent(client=client, tools=toolbox) + server = _make_server(agent, session_store=SqliteSessionStore(tmp_path / "consent.sqlite")) + response = await _post_json(server, {"input": "go", "store": False, "stream": stream}) + + assert response.status_code == 200 + assert CONSENT_LINK not in response.text + assert "oauth_consent_request" not in response.text + assert "requires store=true" in response.text + if stream: + events = _parse_sse_events(response.text) + assert "response.failed" in _sse_event_types(events) + else: + assert response.json()["status"] == "failed" + assert gateway.calls == ["secure"] + + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("link", [CONSENT_LINK, "http://consent.example.com/authorize?state=secret"]) + async def test_disallowed_consent_fails_without_exposing_link( + self, stream: bool, link: str, tmp_path: Path + ) -> None: + gateway = ToolboxConsentServer(["secure", "after"]) + gateway.require_consent("secure", message=f"Request failed (remote): {link}") + async with consent_toolbox(gateway) as toolbox: + client = BatchToolCallClient([function.name for function in toolbox.functions]) + client.function_invocation_configuration["allow_concurrent_invocation"] = False # type: ignore[index] + agent = Agent(client=client, tools=toolbox) + server = _make_server( + agent, + session_store=SqliteSessionStore(tmp_path / "rejected-consent.sqlite"), + allowed_oauth_consent_origins=["https://allowed.example.com"], + ) + response = await _post(server, stream=stream) + + assert response.status_code == 200 + assert link not in response.text + assert "oauth_consent_request" not in response.text + assert "OAuth consent link rejected" in response.text + if stream: + assert "response.failed" in _sse_event_types(_parse_sse_events(response.text)) + else: + assert response.json()["status"] == "failed" + assert gateway.calls == ["secure"] + + def _make_consent_error( url: str = "https://consent.example.com/auth", name: str = "Foundry Toolbox", @@ -8683,16 +9125,26 @@ async def test_explicit_none_origin_allowlist_accepts_any_safe_https_consent(sel oauth_items = [item for item in body["output"] if item["type"] == "oauth_consent_request"] assert [item["consent_link"] for item in oauth_items] == ["https://external.example/authorize"] - async def test_configured_origin_allowlist_accepts_connect_time_consent(self) -> None: + @pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) + @pytest.mark.parametrize("stream", [False, True]) + async def test_configured_origin_allowlist_accepts_connect_time_consent( + self, history_source: str, stream: bool + ) -> None: agent = _make_agent( response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("hi")])]) ) agent.__aenter__.side_effect = _make_consent_error("https://auth.example.com/authorize?state=1") - server = _make_server(agent, allowed_oauth_consent_origins=["https://auth.example.com"]) + agent.client.STORES_BY_DEFAULT = history_source == "inference_service" + server = _make_server( + agent, allowed_oauth_consent_origins=["https://auth.example.com"], history_source=history_source + ) - resp = await _post(server, input_text="hello", stream=False) + resp = await _post(server, input_text="hello", stream=stream) - assert resp.json()["status"] == "incomplete" + if stream: + assert "response.incomplete" in _sse_event_types(_parse_sse_events(resp.text)) + else: + assert resp.json()["status"] == "incomplete" async def test_configured_origin_allowlist_rejects_connect_time_consent( self, caplog: pytest.LogCaptureFixture @@ -9583,5 +10035,38 @@ async def _drain() -> list[Any]: # The in-flight input read must have been cancelled by the handler's cleanup, not orphaned. await asyncio.wait_for(sibling_cancelled.wait(), timeout=1) + async def test_session_preparation_cancellation_drains_reads_and_propagates(self) -> None: + reader = GatedRequestRead() + store = GatedSessionLoadStore(reader.started) + agent = _make_agent() + server = _make_server(agent, session_store=store) + request = CreateResponse(input="hi", previous_response_id="previous", stream=True) + context = ResponseContext(response_id="response-cancel", mode_flags=MagicMock()) + events: list[Any] = [] + + with ( + patch.object(ResponseContext, "get_input_items", new=AsyncMock(side_effect=reader.load)), + patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), + ): + task = asyncio.create_task( + collect_host_events( + cast(AsyncIterator[Any], server._handle_response(request, context, asyncio.Event())), + events, + ) + ) + try: + await asyncio.wait_for(store.started.wait(), timeout=2) + task.cancel() + with pytest.raises(asyncio.CancelledError): + await task + finally: + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + + assert reader.stopped.is_set() + assert store.stopped.is_set() + agent.run.assert_not_called() + # endregion diff --git a/python/packages/foundry_hosting/tests/test_responses_int.py b/python/packages/foundry_hosting/tests/test_responses_int.py index 8fc83283720..69dcb35abce 100644 --- a/python/packages/foundry_hosting/tests/test_responses_int.py +++ b/python/packages/foundry_hosting/tests/test_responses_int.py @@ -45,7 +45,7 @@ ) from agent_framework.foundry import FoundryChatClient, FoundryChatOptions from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled -from azure.ai.agentserver.responses import InMemoryResponseProvider, ResponsesServerOptions +from azure.ai.agentserver.responses import InMemoryResponseProvider from azure.identity import AzureCliCredential from openai import AsyncOpenAI, DefaultAsyncHttpxClient @@ -1069,7 +1069,7 @@ async def parse_input(request: ResponsesRequest) -> WorkflowTurn[list[Message]]: server = ResponsesWorkflowHostServer( workflow=lambda request: build_workflow(sleep_seconds), parse_input=parse_input, - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, ) server.run(host="127.0.0.1", port=port) diff --git a/python/packages/foundry_hosting/tests/test_state_store.py b/python/packages/foundry_hosting/tests/test_state_store.py index 5c622ca1e75..de8afe76b5c 100644 --- a/python/packages/foundry_hosting/tests/test_state_store.py +++ b/python/packages/foundry_hosting/tests/test_state_store.py @@ -6,10 +6,12 @@ from unittest.mock import AsyncMock, MagicMock, patch import pytest -from agent_framework import AgentSession, Content, WorkflowCheckpoint, WorkflowCheckpointException +from agent_framework import AgentSession, WorkflowCheckpoint, WorkflowCheckpointException from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext -from azure.ai.agentserver.core.storage import FoundryStateStore, FoundryStorageConflictError +from azure.ai.agentserver.core.storage import ( + FoundryStateStore, +) from agent_framework_foundry_hosting import ContextScopedStoreProvider, StoreProvider from agent_framework_foundry_hosting._state_store import ( @@ -17,8 +19,6 @@ CheckpointStoreProvider, FoundryAgentSessionStore, FoundryCheckpointStore, - FoundryFunctionApprovalStore, - FunctionApprovalStoreProvider, ) @@ -82,7 +82,6 @@ def _platform_context( def test_storage_providers_use_public_abstraction() -> None: assert issubclass(CheckpointStoreProvider, ContextScopedStoreProvider) assert not issubclass(CheckpointStoreProvider, StoreProvider) - assert issubclass(FunctionApprovalStoreProvider, StoreProvider) assert issubclass(AgentSessionStoreProvider, StoreProvider) @@ -387,95 +386,6 @@ def test_checkpoint_stores_require_context_id(create_store: Callable[[], Any]) - create_store() -def _approval_request(approval_request_id: str) -> Content: - function_call = Content.from_function_call( - "call-1", - "delete_file", - arguments='{"path": "/foo"}', - additional_properties={"server_label": "my_server"}, - ) - return Content.from_function_approval_request(approval_request_id, function_call) - - -async def test_save_and_load_function_approval_request() -> None: - store = _store() - request = _approval_request("approval-1") - store.get_item = AsyncMock(return_value=SimpleNamespace(value=request.to_dict())) - - with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", - new=AsyncMock(return_value=store), - ) as get_or_create: - storage = FoundryFunctionApprovalStore(_platform_context()) - await storage.save_approval_request("approval-1", request) - loaded = await storage.load_approval_request("approval-1") - - assert get_or_create.await_count == 2 - get_or_create.assert_awaited_with("function_approvals", user_isolation=True) - store.create_item.assert_awaited_once_with("approval-1", request.to_dict(), call_id="call-1") - store.get_item.assert_awaited_once_with("approval-1", call_id="call-1") - assert loaded == request - function_call = loaded.function_call - assert function_call is not None - assert function_call.name == "delete_file" - assert function_call.additional_properties["server_label"] == "my_server" - - -async def test_save_duplicate_function_approval_request_raises() -> None: - store = _store() - store.create_item = AsyncMock(side_effect=FoundryStorageConflictError("already exists")) - - with ( - patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", - new=AsyncMock(return_value=store), - ), - pytest.raises(ValueError, match="Approval request with ID 'approval-1' already exists"), - ): - await FoundryFunctionApprovalStore(_platform_context()).save_approval_request( - "approval-1", _approval_request("approval-1") - ) - - -async def test_load_missing_function_approval_request_raises() -> None: - store = _store() - store.get_item = AsyncMock(return_value=None) - - with ( - patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", - new=AsyncMock(return_value=store), - ), - pytest.raises(KeyError, match="Approval request with ID 'missing' does not exist"), - ): - await FoundryFunctionApprovalStore(_platform_context()).load_approval_request("missing") - - -@pytest.mark.parametrize("is_hosted", [True, False]) -def test_function_approval_storage_provider_uses_foundry_store(is_hosted: bool) -> None: - provider = FunctionApprovalStoreProvider() - config = _config(is_hosted=is_hosted) - platform_context = _platform_context() - - storage = provider.get_store(config=config, platform_context=platform_context) - - assert type(storage) is FoundryFunctionApprovalStore - assert storage.platform_context is platform_context - - -def test_function_approval_storage_provider_creates_request_scoped_storage() -> None: - with patch("agent_framework_foundry_hosting._state_store.FoundryFunctionApprovalStore") as storage_type: - provider = FunctionApprovalStoreProvider() - config = _config(is_hosted=False) - first_context = _platform_context("call-1") - second_context = _platform_context("call-2") - provider.get_store(config=config, platform_context=first_context) - provider.get_store(config=config, platform_context=second_context) - - assert storage_type.call_args_list[0].args == (first_context,) - assert storage_type.call_args_list[1].args == (second_context,) - - async def test_set_agent_session_uses_scoped_store() -> None: store = _store() session = AgentSession(session_id="agent-session-1") @@ -631,7 +541,7 @@ def test_agent_session_storage_provider_creates_request_scoped_storage() -> None assert storage_type.call_args_list[1].args == (second_context,) -@pytest.mark.parametrize("provider", ["agent", "checkpoint", "approval"]) +@pytest.mark.parametrize("provider", ["agent", "checkpoint"]) @pytest.mark.parametrize( ("platform_session_id", "request_session_id", "user_id", "call_id"), [ @@ -649,10 +559,8 @@ def test_hosted_store_providers_reject_missing_platform_identity( with pytest.raises(RuntimeError, match="FOUNDRY_AGENT_SESSION_ID|does not match|user ID and call ID"): if provider == "agent": AgentSessionStoreProvider().get_store(config=config, platform_context=context) - elif provider == "checkpoint": - CheckpointStoreProvider().get_store(config=config, context_id="workflow", platform_context=context) else: - FunctionApprovalStoreProvider().get_store(config=config, platform_context=context) + CheckpointStoreProvider().get_store(config=config, context_id="workflow", platform_context=context) async def test_hosted_store_names_partition_sandbox_and_forward_call_id() -> None: @@ -678,13 +586,10 @@ async def get_store(name: str, *, user_isolation: bool) -> MagicMock: config=config, context_id="../../conversation", platform_context=context ) await checkpoints.save(_checkpoint("checkpoint-1")) - approvals = FunctionApprovalStoreProvider().get_store(config=config, platform_context=context) - await approvals.save_approval_request("approval-1", _approval_request("approval-1")) - assert len(names) == len(set(names)) == 6 + assert len(names) == len(set(names)) == 4 assert all(name.split("/")[1] == "v2" and len(name) <= 128 for name in names) assert all("user-1" not in name and "sandbox" not in name and "conversation" not in name for name in names) - assert {call.kwargs["call_id"] for call in store.create_item.await_args_list} == {"call-1", "call-2"} assert {call.kwargs["call_id"] for call in store.set_item.await_args_list} == {"call-1", "call-2"} @@ -704,38 +609,29 @@ async def test_hosted_scope_does_not_read_existing_unscoped_agent_session() -> N assert await legacy.get_item("response-1", call_id=context.call_id) is not None -async def test_hosted_scope_does_not_read_existing_unscoped_checkpoint_or_approval() -> None: +async def test_hosted_scope_does_not_read_existing_unscoped_checkpoint() -> None: context = _platform_context() legacy_checkpoint = await FoundryStateStore.get_or_create("checkpoints/conversation-1", user_isolation=True) - legacy_approval = await FoundryStateStore.get_or_create("function_approvals", user_isolation=True) async with legacy_checkpoint: await legacy_checkpoint.create_item( "checkpoint-1", encode_checkpoint_value(_checkpoint("checkpoint-1").to_dict()), call_id=context.call_id ) - async with legacy_approval: - await legacy_approval.create_item( - "approval-1", _approval_request("approval-1").to_dict(), call_id=context.call_id - ) config = _config(is_hosted=True) checkpoints = CheckpointStoreProvider().get_store( config=config, context_id="conversation-1", platform_context=context ) - approvals = FunctionApprovalStoreProvider().get_store(config=config, platform_context=context) with pytest.raises(WorkflowCheckpointException, match="No checkpoint found"): await checkpoints.load("checkpoint-1") - with pytest.raises(KeyError, match="does not exist"): - await approvals.load_approval_request("approval-1") -async def test_hosted_session_checkpoint_and_approval_data_stay_in_one_sandbox() -> None: +async def test_hosted_session_and_checkpoint_data_stay_in_one_sandbox() -> None: config = _config(is_hosted=True) first_context = _platform_context(session_id="sandbox-1") second_context = _platform_context(session_id="sandbox-2") second_config = _config(is_hosted=True, session_id="sandbox-2") sessions = AgentSessionStoreProvider() checkpoints = CheckpointStoreProvider() - approvals = FunctionApprovalStoreProvider() first_session_store = sessions.get_store(config=config, platform_context=first_context) await first_session_store.set("response-1", AgentSession()) @@ -759,15 +655,6 @@ async def test_hosted_session_checkpoint_and_approval_data_stay_in_one_sandbox() with pytest.raises(WorkflowCheckpointException, match="No checkpoint found"): await second_checkpoint_store.load("checkpoint-1") - first_approval_store = approvals.get_store(config=config, platform_context=first_context) - await first_approval_store.save_approval_request("approval-1", _approval_request("approval-1")) - continued_approval_store = approvals.get_store(config=config, platform_context=continued_context) - assert await continued_approval_store.load_approval_request("approval-1") == _approval_request("approval-1") - second_approval_store = approvals.get_store(config=second_config, platform_context=second_context) - with pytest.raises(KeyError, match="does not exist"): - await second_approval_store.load_approval_request("approval-1") - assert await first_approval_store.load_approval_request("approval-1") == _approval_request("approval-1") - async def test_hosted_session_rejects_a_stale_etag_without_overwriting_the_winner() -> None: config = _config(is_hosted=True) diff --git a/python/packages/foundry_hosting/tests/test_toolbox.py b/python/packages/foundry_hosting/tests/test_toolbox.py index 6396df53df1..166a6fe18d9 100644 --- a/python/packages/foundry_hosting/tests/test_toolbox.py +++ b/python/packages/foundry_hosting/tests/test_toolbox.py @@ -9,6 +9,7 @@ import json from collections.abc import Awaitable, Callable from datetime import datetime, timedelta, timezone +from pathlib import Path from types import SimpleNamespace from typing import Any, cast from unittest.mock import AsyncMock, Mock, patch @@ -44,7 +45,11 @@ _ToolboxAuth, ) +from .foundry_hosting.approval_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + SqliteSessionStore, +) from .foundry_hosting.toolbox_approval_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + FixedSessionStoreProvider, ToolboxApprovalClient, ToolboxApprovalServer, toolbox_connection, @@ -1494,12 +1499,15 @@ async def test_metadata_approval_pauses_and_resumes_once(stream: bool, approved: @pytest.mark.parametrize("stream", [False, True]) -async def test_responses_host_returns_metadata_approval_before_tool_execution(stream: bool) -> None: +async def test_responses_host_returns_metadata_approval_before_tool_execution(stream: bool, tmp_path: Path) -> None: mcp_server = ToolboxApprovalServer() mcp_server.tools = [toolbox_tool()] model = ToolboxApprovalClient("learn___microsoft_docs_search") async with toolbox_connection(mcp_server) as toolbox: - host = ResponsesHostServer(agent=Agent(client=model, tools=toolbox)) + host = ResponsesHostServer( + agent=Agent(client=model, tools=toolbox), + agent_session_store_provider=FixedSessionStoreProvider(SqliteSessionStore(tmp_path / "sessions.db")), + ) async with host_httpx.AsyncClient( transport=host_httpx.ASGITransport(app=host), base_url="http://test" ) as client: diff --git a/python/samples/04-hosting/foundry-hosted-agents/README.md b/python/samples/04-hosting/foundry-hosted-agents/README.md index db81091f2dd..abfc055efe7 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/README.md @@ -4,7 +4,12 @@ This directory contains samples that demonstrate how to use hosted [Agent Framew > [IMPORTANT] Migrating from Protocol version 1.0.0 to 2.0.0: Foundry Hosting Agents service has been updated to use Protocol version 2.0.0. If your application is using Protocol version 1.0.0, please upgrade to Protocol version 2.0.0 in your `agent.manifest.yaml` or `agent.yaml` and upgrade to the latest `agent-framework-foundry-hosting` package. `agent-framework-foundry-hosting==1.0.0a260625` is the last version that supports Protocol version 1.0.0. > -> The `agent-framework-foundry-hosting` Python API surface is intended to remain stable, but protocol 1.0.0 and 2.0.0 are incompatible. +> Protocol versions 1.0.0 and 2.0.0 are incompatible. + +The hosting package is currently in beta, so its public APIs are experimental. Responses hosting +is the target for an initial stable release, subject to qualification. Invocations hosting and +the `FoundryToolbox` integration remain experimental. + ## Samples ### Responses API @@ -31,8 +36,11 @@ The integration examples use **request-owned factories** and the current `history_source="host_managed"` API. Use the current workspace, or a hosting release containing that API; before the coordinated beta is published, older PyPI wheels are not evidence that the examples include these behaviors. -Toolbox/skills connections inherit the current request's call ID and are closed -afterward. Files and custom MAF state are sandbox-specific; Foundry Memory +Toolbox/skills connections inherit the current request's call ID; entered +contexts are unwound after the request. Pre-entry allocations and partial +allocations from failed client entry may not be cleaned up. See +[factory resource lifecycle](../../../packages/foundry_hosting/README.md#factory-resource-lifecycle). +Files and custom MAF state are sandbox-specific; Foundry Memory intentionally shares long-term memories across the **same user's** sandboxes. Their READMEs document the external resource/permission boundaries. GitHub MCP PAT/OAuth and Telegram/Key Vault require separate setup; the @@ -74,6 +82,10 @@ An opaque call ID correlates one request; it is not a storage namespace. ### Invocations API +The Foundry-hosted Telegram integration is sample-only, not a built-in channel adapter in +`agent-framework-foundry-hosting`. The sample uses Azure AgentServer's `InvocationAgentServerHost` +directly, not Agent Framework's `InvocationsHostServer`. + | # | Sample | Description | |---|--------|-------------| | 1 | [Basic](invocations/basic/) | An Invocations agent with a custom JSON parser, durable MAF history, and JSON/SSE responses. | diff --git a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md index a5bf15b3ba9..ff1f08419d0 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md @@ -1,10 +1,14 @@ # Invocations agent with a custom request parser +Agent Framework Invocations hosting is experimental. + [`main.py`](main.py) hosts an Agent Framework agent using the Foundry Invocations protocol. The sample accepts application JSON with a `prompt` field rather than the host's default `message` field. Its `parse_input` callback returns `InvocationRun(messages, options, stream)`, and `prepare_options` allows only `temperature` and `max_tokens` from the caller. +Only the hook's returned, validated mapping is forwarded. Without a hook, caller +model-run options are suppressed and the agent/client defaults apply. The host also rejects platform IDs, `store`, and private continuation options even if a hook tries to return them. The agent keeps `store=False` as a developer default; caller options cannot enable service-managed history. @@ -28,7 +32,7 @@ curl -i -X POST http://localhost:8088/invocations \ -d '{"prompt": "Hi", "options": {"temperature": 0.3}}' ``` -The default wire format is JSON, for example: +Non-streaming success is JSON, for example: ```http HTTP/1.1 200 OK @@ -72,11 +76,7 @@ for scope and retention details. Malformed requests return JSON client errors. Streaming failures produce `event: error` instead of `done`; a competing host's session write is reported as a conflict, not silently overwritten. Do not blindly retry calls with -non-idempotent tools. Existing clients that still require the old plain-text -response and raw text-chunk stream can set `legacy_wire_format=True` on the -host temporarily; the host warns once and the mode is deprecated. Migrate -clients to JSON and framed SSE before removing that opt-in in a deliberate -breaking change. +non-idempotent tools. Follow the [parent deployment guide](../../README.md#deploying-the-agent-to-foundry) when deploying this example to Foundry. @@ -150,9 +150,7 @@ If the connection drops after commit, retry the request with the same trusted user, sandbox and platform call ID to replay the complete stored response without executing the ticket workflow again. Failures produce `error`, not `done`. Non-streaming pauses expose the same -`request_info` data in the JSON event list. Native workflows do not support -`legacy_wire_format=True`; the old plain-text/raw-chunk opt-in applies only -to the agent entrypoint. +`request_info` data in the JSON event list. ### Checkpoints, fresh factories and pending replies diff --git a/python/samples/04-hosting/foundry-hosted-agents/invocations/telegram/README.md b/python/samples/04-hosting/foundry-hosted-agents/invocations/telegram/README.md index f33634d418b..5810c11bd9d 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/invocations/telegram/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/invocations/telegram/README.md @@ -15,6 +15,10 @@ It is a complete learning deployment rather than a production reference architec polling, an app-owned FastAPI webhook, and in-memory state. This sample focuses on Foundry direct-code deployment, APIM ingress, managed identities, and durable history. +The Foundry-hosted Telegram integration is sample-only, not a built-in channel adapter in +`agent-framework-foundry-hosting`. This sample uses Azure AgentServer's `InvocationAgentServerHost` +directly, not Agent Framework's `InvocationsHostServer`. + ## What is deployed The self-contained Bicep templates create: diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md index 0209037e016..b7188b5a8ad 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md @@ -15,9 +15,11 @@ The agent uses `FoundryChatClient` from the Agent Framework to create a Response See [main.py](main.py) for the full implementation. `create_agent` allocates a fresh Search provider, credential and model client for -each request. The sample's context-managed client explicitly enters/closes the -provider and closes its own SDK transports and credential, including failed -entry and cancellation; `Agent` does not automatically enter every provider. +each request. Once entered, the sample's client context closes the provider and +its own SDK transports and credential on exit. The client also attempts cleanup +if Search provider entry fails; `Agent` does not automatically enter every +provider. If the request ends before client entry, those allocations may not be +cleaned up. `history_source="host_managed"` owns conversation history and disables inner model storage. Only first-party Foundry model calls receive the platform call ID; it is not forwarded to Search as an authorization substitute. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md index 479856271ce..d2c1e82886e 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md @@ -8,7 +8,7 @@ who supplies the agent's model history: | [main.py](main.py) | `"host_managed"` | The outer Responses transcript; the inner client runs with `store=False`. | | [service_history.py](service_history.py) | `"inference_service"` | Only new input goes to the model; its private `service_session_id` persists for later turns. | | [agent_history.py](agent_history.py) | `"agent_managed"` | This agent opts into `InMemoryHistoryProvider` with `default_options={"store": False}`; another `"agent_managed"` configuration with `store=True` can use service history instead. | -| [options.py](options.py) | `"host_managed"` | The hook removes the caller's token limit so the agent default is used. | +| [options.py](options.py) | `"host_managed"` | The hook removes the effective caller model so the configured client model is used; other generation settings remain. | | [provider_background.py](provider_background.py) | `"inference_service"` | `background_source="inference_service"` opts into provider background with a private recovery token. | For the same two stored requests—first **"My name is Ada"**, then **"What is my name?"** with the first @@ -75,19 +75,41 @@ turns fail rather than mutating the same service thread. An invalid input does n `previous_response_id` parent. If a named turn fails or is cancelled after dispatch, start a new conversation; the existing thread may have changed even if its outer response did not complete. -## Options and compatibility +## Options + +Caller generation options are suppressed unless the host has a `prepare_options` hook. +Without a hook, the agent/client defaults apply. With a hook, only its returned, validated mapping +is forwarded. The options sample explicitly opts in; the other basic hosts do not. Native CreateResponse generation fields become MAF run options (`max_output_tokens` becomes `max_tokens` and -`parallel_tool_calls` becomes `allow_multiple_tool_calls`). Flattened OpenAI `extra_body` values overlay translated -keys **last**. The developer's `prepare_options(request, options)` hook can then remove or replace *caller* options; -removing one exposes the agent's own unchanged `default_options`. In [options.py](options.py), -`max_output_tokens=300` plus `extra_body={"max_tokens": 150}` becomes `max_tokens=150` before the hook removes it; -the model receives the agent's `max_tokens=256` default instead. Platform IDs, storage flags, and private -continuation tokens are never caller model options; the hook cannot add them back. - -`history_source="host_managed"` and `history_source="agent_managed"` retain their existing meaning without a -deprecation warning. For **stored** requests, `"agent_managed"` passes only new input to the agent, whose +`parallel_tool_calls` becomes `allow_multiple_tool_calls`). Flattened unknown generation fields override those +translated fields; generation fields inside a literal `extra_body` mapping override both. The developer's +`prepare_options(request, options)` hook then returns the replacement *caller* options. The host deep-copies +both its input and result, and keeps developer defaults unchanged. + +In [options.py](options.py), the hook removes `"model"` from the effective merged options, so the client uses +the configured `FOUNDRY_MODEL` (or hosted deployment-name fallback). It preserves `"temperature"` and +`"max_tokens"` if supplied. For example: + +```bash +curl -X POST http://localhost:8088/responses -H "Content-Type: application/json" \ + -d '{"input":"Hi","model":"caller-selected-model","temperature":0.3,"max_output_tokens":150}' +``` + +The client uses the developer's model with `temperature=0.3` and `max_tokens=150`, not the caller's model. +If the caller omits a token limit, the agent's `max_tokens=256` default applies. +Responses agent factories see translated caller options before the hook. Native Responses workflow hosts have +no option hook: their factories and required parsers see the translated caller options through `request.options` +and the original request through `request.request`, and the workflow receives model options only from the +parser's `WorkflowTurn(client_kwargs={"options": ...})`. Platform IDs, storage flags, and private +continuation tokens are never caller model options; the hook cannot add them back. Host controls inside +`extra_body` fail before the hook, rather than being silently stripped. +See [explicit host configuration](../../../../../packages/foundry_hosting/README.md#explicit-host-configuration) +for history caps, heartbeat intervals, shutdown waiting, and eligible recovery modes. + +`history_source="host_managed"` keeps outer history in the host. For **stored** requests, +`"agent_managed"` passes only new input to the agent, whose developer-owned defaults may choose either a HistoryProvider **or downstream service storage**. `history_source="inference_service"` explicitly requires service-managed history and overrides the agent's `store` default on stored requests. The constructor accepts `response_store=` for the outer storage -*backend*, not `store=`. The removed alias does not affect the caller's per-request `store` flag. +*backend*; the caller's per-request `store` flag is separate. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py index 681e27d1849..ec415b6c868 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py @@ -16,8 +16,8 @@ def prepare_options(_request: ResponsesRequest, options: dict[str, Any]) -> dict[str, Any]: - """Use the agent's output-token default instead of the caller's override.""" - options.pop("max_tokens", None) + """Keep the developer's model while preserving other caller generation settings.""" + options.pop("model", None) return options diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py index 67e46fe0b7c..1712885b910 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py @@ -7,7 +7,6 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest -from azure.ai.agentserver.responses import ResponsesServerOptions from azure.identity import DefaultAzureCredential from dotenv import load_dotenv @@ -31,7 +30,7 @@ def main() -> None: agent=create_agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md index 3e88df652e0..48b94922c29 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md @@ -1,8 +1,10 @@ # Custom session storage (Responses protocol) This sample implements custom **MAF agent-session persistence**, including user -and sandbox isolation and optimistic concurrency. The outer Responses store and -function-approval store remain host-managed. This is not a workflow example. +and sandbox isolation and optimistic concurrency. The outer Responses store +remains host-managed. Pending tool approvals are part of the saved +`AgentSession`, so the store needs only `get`/`set`/`delete`. This is not a +workflow example. ## Identity and storage keys @@ -51,10 +53,14 @@ newer snapshot. An absent delete is idempotent. Missing ETags and mismatched stored scope fail closed; other Cosmos failures propagate without a local fallback. -Local runs use isolated in-memory snapshots with the same conditional behavior. -They are lost when the process exits and are **single-user development only**: -local headers/session selectors are not a trusted platform authentication -boundary. Explicit local sessions have separate namespaces. +For local runs, each `CustomSessionStoreProvider` owns its in-memory snapshots. +Only the stores from that provider in **one process and event loop** coordinate +their conditional writes. Separate provider instances do not share snapshots, +even in the same process. Use one provider and one worker for local development. +Snapshots are lost when the provider and its stores are discarded or the +process exits. They are **single-user development only**: local headers and +session selectors are not a trusted platform authentication boundary. Explicit +local sessions have separate namespaces. ## Cosmos prerequisites @@ -94,9 +100,10 @@ does not access/write this custom session store. For hosted use, configure the endpoint/database/container in the manifest and follow the [parent deployment instructions](../../README.md#deploying-the-agent-to-foundry). -The provider closes the Cosmos client and its own credential at shutdown; -request clients are closed after their request, including failed tool entry and -cancellation. +The provider closes the Cosmos client and its own credential at shutdown. +After a request client enters, its context closes its own transports and +credential on exit, including cancellation. If a request ends before client +entry, those allocations may not be cleaned up. Live Cosmos access and deployment need separately approved resources and permissions. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py index 0435acc9ef6..c2214351e2d 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py @@ -1,3 +1,5 @@ +# Copyright (c) Microsoft. All rights reserved. + # /// script # requires-python = ">=3.11" # dependencies = [ @@ -12,8 +14,6 @@ # ] # /// -# Copyright (c) Microsoft. All rights reserved. - """Trusted user-and-sandbox session snapshots with optimistic concurrency. Hosted runs use an existing Cosmos DB container partitioned by /scope_key and @@ -144,7 +144,11 @@ async def delete(self, session_id: str) -> None: class LocalSessionStore(SessionStore): - """Request-local concurrency tokens over scoped, process-local snapshots.""" + """Request-local concurrency tokens over scoped, process-local snapshots. + + Conditional writes coordinate only stores sharing the same snapshots in one + event loop. Separate providers, workers, and processes do not share them. + """ def __init__(self, snapshots: dict[tuple[str, str], tuple[str, dict[str, Any]]], *, scope_key: str) -> None: super().__init__() @@ -182,7 +186,7 @@ async def delete(self, session_id: str) -> None: class CustomSessionStoreProvider(StoreProvider[SessionStore]): - """Reuse only the backend; each request gets an isolated ETag/working-copy view.""" + """Share one backend across request-specific ETag and working-copy views.""" def __init__(self) -> None: self._local_snapshots: dict[tuple[str, str], tuple[str, dict[str, Any]]] = {} diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md index a42f6126995..de32bdfbc3a 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md @@ -5,7 +5,7 @@ using `InvokeAzureAgent`, `ConditionGroup`, `SendActivity`, and Power Fx expressions. [`main.py`](main.py) hosts the built declarative `Workflow` directly through `ResponsesWorkflowHostServer(workflow=..., parse_input=...)`. -The parser uses the explicit `response_input_messages()` migration helper for +The parser uses the explicit `response_input_messages()` helper for the declarative workflow's `list[Message]` start contract. The host restores the exact scoped MAF checkpoint first; it does not silently unwrap a `WorkflowAgent`, rebuild outer Responses history as workflow state, or reuse a @@ -13,9 +13,8 @@ downstream service session. Every request-aware factory call creates fresh agents, Foundry clients, credentials, and a fresh graph while the stable IDs in `workflow.yaml` preserve -checkpoint compatibility. Start a fresh Responses chain when migrating from -the legacy `.as_agent()` hosting path because old unscoped checkpoints are not -safe across Foundry sandboxes. +checkpoint compatibility. Unscoped checkpoints are not safe across Foundry +sandboxes, so this host does not read them. Example: diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py index 7a43d92bcb0..17dd061acb0 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py @@ -106,7 +106,7 @@ def build_workflow(request: ResponsesRequest) -> Workflow: async def parse_input(request: ResponsesRequest) -> WorkflowTurn[list[Message]]: - """Use the explicit current-turn message migration helper; checkpoints retain prior workflow state.""" + """Use the explicit current-turn message helper; checkpoints retain prior workflow state.""" return WorkflowTurn(input=await response_input_messages(request)) diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md index 9efc115b127..3405b693610 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md @@ -24,10 +24,10 @@ Set `FOUNDRY_PROJECT_ENDPOINT` and `FOUNDRY_MODEL`, plus either see the [Toolbox sample](../foundry_toolbox/). Authenticate local runs with `az login`. Deployed runs use the sandbox's managed identity. -`ResponsesHostServer(agent=create_agent, history_source="host_managed")` creates -fresh clients and a Toolbox MCP connection for each request and closes their -transports afterward. This request-owned lifecycle gives the sample a clear -cleanup boundary; `FoundryToolbox` also rebinds its MCP session to current +`ResponsesHostServer(agent=create_agent, history_source="host_managed")` uses a +factory that creates a fresh client and Toolbox configuration for the request. +Successfully entered contexts are unwound after the request; pre-entry allocations +may not be cleaned up. `FoundryToolbox` also rebinds its MCP session to current platform headers if an instance is reused across requests. The outer Responses service supplies history; the host disables downstream model storage. An outer `store=false` request writes no host-managed state, but does not undo external diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md index 315e3d92308..61f7ed9a260 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md @@ -20,9 +20,9 @@ The request-aware `create_agent` factory builds a new agent, credential, project client and Memory provider **inside each request**. Chat and Memory share that request's project client, not a process-wide client. Its headers capture the current platform call ID for both Memory and model calls; raw user IDs are not -forwarded. The request-owned client's context manager closes its own OpenAI and -project transports and credential on completion or cancellation. It does not -change ownership of developer-supplied clients elsewhere in the framework. +forwarded. Once entered, the request-owned client context closes its own OpenAI +and project transports and credential on completion or cancellation. Host cleanup +may not cover allocations when a request ends before client entry. The Memory namespace is a framed hash of the **trusted platform user ID**. It intentionally excludes the sandbox, conversation, MAF session and call IDs: the diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md index 1d1962379c8..179f0a7ecdf 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md @@ -2,6 +2,8 @@ An [Agent Framework](https://github.com/microsoft/agent-framework) agent that uses **Foundry Toolbox** for tool discovery, hosted on Microsoft Foundry using the **Responses protocol**. Foundry Toolbox is a managed tool registry in Microsoft Foundry that lets you define tools centrally and share them across agents. +The Agent Framework `FoundryToolbox` integration is experimental. + ## Creating a Foundry Toolbox You can create a Foundry Toolbox by code. Refer to this sample for an example: [Foundry Toolbox CRUD Sample](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-projects/samples/toolboxes/sample_toolboxes_crud.py). @@ -98,16 +100,17 @@ See [main.py](main.py) for the full implementation. ### Request-owned resources The host receives `agent=create_agent`, not a process-wide agent instance. -Every request creates a fresh client and Toolbox MCP connection, then closes -them afterward. This gives the sample one clear owner for its model transports, -credential and Toolbox lifecycle. `FoundryToolbox` also supports long-lived -instances: it resolves the current platform headers before each operation and +The factory creates a fresh client and Toolbox configuration for the request. +Successfully entered contexts are unwound after the request. `FoundryToolbox` +also supports long-lived instances: it resolves the current platform headers before each operation and reconnects the MCP session when that identity changes, so a shared instance does not retain an earlier caller's `x-agent-foundry-call-id`. -The sample client closes only its own model/project transports and credential, -including when MCP entry fails or the request is cancelled. Local authentication -uses `AzureCliCredential`; deployed authentication uses managed identity. +After entry, the sample client context closes its own model/project transports +and credential on exit, including if later MCP entry fails or the request is +cancelled. If the request ends before client entry, those allocations may not +be cleaned up. Local authentication uses `AzureCliCredential`; deployed +authentication uses managed identity. `history_source="host_managed"` reconstructs the outer Responses transcript and disables inner model storage. Caller `store=false` prevents host-managed state, not external tool side effects. @@ -117,6 +120,9 @@ PAT-backed tools act as the configured external account; user-identity tools require the appropriate pass-through connection and the current call ID. A fresh Python object does not by itself grant external permissions. GitHub PAT/OAuth connections, project roles and consent are separate setup, not offline coverage. +If a Toolbox tool call needs OAuth consent, the response is `incomplete` and +contains an `oauth_consent_request`. Continuation needs `store=true`. See +[Invocation-time OAuth consent](../../../../../packages/foundry_hosting/README.md#invocation-time-oauth-consent). Native code-interpreter file citations are a separate tracked behavior (microsoft/agent-framework#7916), not implemented by this lifecycle change. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md index 509ad01d419..01290146ae5 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md @@ -2,6 +2,8 @@ An [Agent Framework](https://github.com/microsoft/agent-framework) agent that discovers **Agent Skills attached to a Foundry Toolbox** over the **MCP protocol** and exposes them to the model using the [Agent Skills](https://agentskills.io/) progressive-disclosure pattern, hosted on Microsoft Foundry using the **Responses protocol**. +The Agent Framework `FoundryToolbox` integration is experimental. + This sample is **self-contained**: it ships the `SKILL.md` sources and a `toolbox.yaml`, and walks you through creating the skills and the toolbox from zero with `azd` — you don't need an existing toolbox to run it. ## How progressive disclosure works @@ -31,18 +33,20 @@ The `FoundryToolbox` is attached to the agent and its skills are exposed through The agent is hosted with the `ResponsesHostServer`, which provisions a REST API endpoint compatible with the OpenAI Responses protocol on `http://localhost:8088`. -`agent=create_agent` builds a new Toolbox, skills provider/cache, credential and -client for **each request**, giving those resources one deterministic owner and -cleanup boundary. `FoundryToolbox` also scopes tool and skill-resource +The `create_agent` factory builds a new Toolbox, skills provider/cache, credential +and client for the request. Successfully entered contexts are unwound after the +request. `FoundryToolbox` also scopes tool and skill-resource operations to the current platform call ID when an instance is reused, rebinding its MCP session when that identity changes. `tools=` still connects the Toolbox and `context_providers=` still reads skills from that same session; both are required. Only loading the vetted sample skill body is auto-approved, not arbitrary skill scripts or external actions. -The factory's client context closes its owned SDK transports and credential -after completion, failed tool entry or cancellation. Local runs use -`AzureCliCredential`; hosted runs use managed identity. +After entry, the factory's client context closes its owned SDK transports and +credential on exit, including if later tool entry fails or the request is +cancelled. If the request ends before client entry, those allocations may not +be cleaned up. +Local runs use `AzureCliCredential`; hosted runs use managed identity. `history_source="host_managed"` owns conversation history and disables inner model storage. Scope the configured Toolbox/skill resources and RBAC to the intended project; a fresh provider is not an external authorization boundary. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md index bd619918241..b9ca92e049a 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md @@ -10,8 +10,9 @@ The agent uses `FoundryChatClient` from the Agent Framework to create an OpenAI- See [main.py](main.py) for the full implementation. -`agent=create_agent` creates fresh client and hosted-MCP tool configuration for -each request and closes the request's own SDK transports and credential. +The `create_agent` factory creates fresh client and hosted-MCP tool configuration +for the request. Once entered, the client context closes its own SDK transports +and credential on exit; pre-entry allocations may not be cleaned up. Local model authentication uses `AzureCliCredential`; deployed calls use managed identity. `history_source="host_managed"` supplies the outer transcript and disables inner model storage. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md index 7bf5d5fb7e8..0950921ab37 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md @@ -24,8 +24,9 @@ See [main.py](main.py) for the full implementation. `ResponsesHostServer(agent=create_agent, history_source="host_managed")` constructs a fresh CodeAct provider, client and credential for each request. No interpreter/provider instance, tool connection or model client is reused -from an earlier caller. The factory's client context closes its owned SDK -transports and credential after the request. Local calls use +from an earlier caller. Once entered, the factory's client context closes its +owned SDK transports and credential on exit; pre-entry allocations may not be +cleaned up. Local calls use `AzureCliCredential`; hosted calls use managed identity. The host supplies conversation history and disables inner model storage. The diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md index 4801a2ac3fd..43a0a186ee6 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md @@ -25,8 +25,9 @@ a production default. Never log platform identity values or tokens. The host receives `agent=create_agent` and builds a new client/credential for each request, capturing only that request's first-party platform headers. Native OpenTelemetry context propagation links spans to the incoming request rather -than a preceding caller. The context-managed sample client closes its own SDK -transports and credential after the request. Local authentication uses +than a preceding caller. Once entered, the sample client context closes its own +SDK transports and credential on exit; pre-entry allocations may not be cleaned +up. Local authentication uses `AzureCliCredential`; hosted authentication uses managed identity. `history_source="host_managed"` owns the outer transcript and disables inner model storage. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md index c9d44c19218..6e45d0fc884 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md @@ -27,8 +27,8 @@ Do not delete shared state as part of a sample or test. Use an isolated `AGENTSERVER_STATE_ROOT` when experimenting with crashes. The Foundry sandbox, the Responses continuation, and the MAF checkpoint remain -separate identities. Existing legacy unscoped workflow sessions are not read; -start a fresh response chain when migrating from `agent=workflow.as_agent()`. +separate identities. Unscoped workflow sessions are not read; +start a fresh response chain for this host. For Foundry deployment instructions, see the [parent README](../../README.md#deploying-the-agent-to-foundry). diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py index a95415d83ce..2e3b2754430 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py @@ -15,7 +15,6 @@ WorkflowTurn, ) from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled -from azure.ai.agentserver.responses import ResponsesServerOptions from pydantic import BaseModel, Field @@ -72,7 +71,7 @@ def main() -> None: checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{CountdownRequest.__module__}:{CountdownRequest.__qualname__}"], ), - options=ResponsesServerOptions(resilient_background=True), + enable_background_recovery=True, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md index f0b2f5526fa..477bf260513 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md @@ -1,7 +1,7 @@ # Long-running Responses agent (steering temporarily unavailable) -**Steering is currently gated.** `ResponsesHostServer` rejects -`ResponsesServerOptions(steerable_conversations=True)` during construction rather than starting a +**Steering is currently gated.** `ResponsesHostServer` has no +`steerable_conversations` setting and raises `TypeError` during construction rather than starting a TaskManager that can retain unbounded futures after rejected steering requests. The upstream fix is [Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233); steering will be re-enabled only after a patched official `azure-ai-agentserver-core` wheel is released, diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py index 8169f5769b8..c4eac32b065 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py @@ -4,7 +4,7 @@ The agent is asked to count down from a target number, pacing its own output with a short remark before each number so a response takes a while to fully generate. This sample currently runs -without steering; the hosting layer rejects ``steerable_conversations=True`` until a patched +without steering; the hosting layer does not accept a ``steerable_conversations`` setting until a patched AgentServer SDK is released and verified. Environment variables: diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py index f5c6b2f89f1..852f46c67fd 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py @@ -10,7 +10,6 @@ from agent_framework import SupportsAgentRun from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest -from azure.ai.agentserver.responses import ResponsesServerOptions def _agent_factory(*, request: ResponsesRequest) -> SupportsAgentRun: @@ -20,13 +19,13 @@ def _agent_factory(*, request: ResponsesRequest) -> SupportsAgentRun: def main() -> None: """Confirm that the constructor rejects steering before creating an agent or host.""" try: - ResponsesHostServer(agent=_agent_factory, options=ResponsesServerOptions(steerable_conversations=True)) - except RuntimeError as exc: - if "steerable_conversations=True is temporarily unavailable" not in str(exc): + ResponsesHostServer(agent=_agent_factory, steerable_conversations=True) # type: ignore[call-arg] + except TypeError as exc: + if "steerable_conversations" not in str(exc): raise print("Steering is temporarily unavailable; no agent or host was started.") return - raise RuntimeError("Steering guard was removed; restore end-to-end and queue-overflow verification.") + raise RuntimeError("The host accepted steering; restore end-to-end and queue-overflow verification.") if __name__ == "__main__": diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md index 49a1d4b3da0..f3b138aa886 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md @@ -24,9 +24,11 @@ When a tool is set to `always_require`, the agent host emits an `mcp_approval_re The agent is hosted using the [Agent Framework](https://github.com/microsoft/agent-framework) with the `ResponsesHostServer`, which provisions a REST API endpoint compatible with the OpenAI Responses protocol. -The host uses `agent=create_agent` and `history_source="host_managed"`. Each -request receives its own client and credential; their owned transports are -closed afterward. History and approval continuation remain in the host's +The host uses `agent=create_agent` and `history_source="host_managed"`. The +factory creates a client and credential for the request. Once entered, the client +context closes its owned transports on exit; pre-entry allocations may not be +cleaned up. +History and approval continuation remain in the host's trusted user/sandbox-scoped stores, not on a reused agent instance. A fresh factory does not remove pending approvals: return the approval response with the same outer continuation and sandbox. `store=false` cannot establish a