diff --git a/docs/decisions/0055-python-foundry-hosting-contracts.md b/docs/decisions/0055-python-foundry-hosting-contracts.md index 2997fc3f34f..30c76e5be7d 100644 --- a/docs/decisions/0055-python-foundry-hosting-contracts.md +++ b/docs/decisions/0055-python-foundry-hosting-contracts.md @@ -46,8 +46,10 @@ Agent hosts reject `WorkflowAgent`; workflow hosts require a factory returning a Do not silently unwrap agents, migrate their state, or retain a combined-host compatibility shim. Workflow hosts expose checkpoint storage rather than agent-session storage or agent history settings. -The Responses workflow host owns native execution directly. Share the Invocations engine between its -public hosts rather than duplicating it. Group each protocol into agent, workflow, and shared modules, +Both Invocations hosts inherit directly from the SDK host and own their mode-specific construction, +parsing, state, and execution. Do not retain a private combined host or replace it with a mode-switching +base or mixin. Share small identity, wire, and streaming-cleanup helpers rather than duplicating protocol +logic. Group each protocol into agent, workflow, and shared modules, with cross-protocol request contracts and factory resolution in a central shared module. Keep all hosts in one package. Responses is the initial stable-surface target; Invocations and Toolbox @@ -113,15 +115,23 @@ Recovery permission neither selects an execution mode nor guarantees exactly-onc ### Supported host settings Expose supported first-class settings and construct SDK options internally, without another framework -options wrapper. Reject inapplicable or removed settings before side effects; do not leak them through -`**kwargs` to the SDK. +options wrapper. Validate framework-owned settings and their applicability before side effects. +Remove variadic constructor `**kwargs` from all four public hosts. Unknown top-level names fail through +Python argument binding, including removed names supplied as `None`. + +Expose `agent_server_kwargs: Mapping[str, Any] | None = None` as a separate, caller-owned SDK boundary. +Expand its entries only into the upstream SDK constructor, without inspecting, validating, filtering, +or mutating the contents. Let Python and the SDK report invalid arguments and duplicate keyword +collisions; do not add a reserved-key blacklist or a custom merge policy. This does not remove validation +of framework parameters, caller model options, hooks, or execution policies. | Setting | Contract | | --- | --- | +| `log_level=None` | Available on all four hosts as `Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] \| None`, forwarded unchanged to the SDK constructor. `None` preserves the SDK default. | | `max_history_items: int \| None = None` | `None` supplies no framework override and inherits the installed supported SDK's default. Positive caps count prior history items and apply only to regular host-managed agent history. | -| `sse_keep_alive_interval_seconds: int \| None = None` | Use an explicit positive interval, otherwise inherit a positive `SSE_KEEPALIVE_INTERVAL`, otherwise disable heartbeats. No separate force-disable switch. | +| `sse_keep_alive_interval_seconds: int \| None = None` | Available on all four framed-SSE hosts. Explicit values must be positive; zero and negative values are invalid. `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise disables heartbeats. No separate force-disable switch. | | `shutdown_grace_period_seconds: int = 10` | Shutdown waiting, not a per-request timeout. Also document the upstream reuse for DELETE waiting on response persistence. | -| `enable_background_recovery: bool = False` | Permit recovery of eligible background work without selecting background execution. | +| `resilient_background: bool = False` | Permit recovery of eligible background work without selecting background execution. | Do not interpret a history limit of `None` as unlimited, map it to `-1`, or raise the dependency floor to obtain a particular SDK default. History caps do not limit tokens, turns, retention, or checkpoints. @@ -166,6 +176,7 @@ These decisions do not redesign Core, concurrency, checkpoints, retries, or exte - Shared internals avoid duplicate engines without obscuring public contracts. - Request-aware factories support identity-dependent resources but require deliberate isolation and cleanup. - The API migration breaks combined-host construction, wrapper hosting, unnamed or zero-argument agent factories, old parser/request names, source literals, and the storage-backend alias. +- SDK constructor arguments move from arbitrary top-level keywords to the explicit, unvalidated `agent_server_kwargs` mapping. The public SDK options bag is replaced by supported first-class settings. - Settings, authority, and consent changes require their own implementation and correctness validation; recording a decision does not establish that it has shipped. ## Validation diff --git a/python/packages/foundry_hosting/README.md b/python/packages/foundry_hosting/README.md index f37178cad2f..e777022ab02 100644 --- a/python/packages/foundry_hosting/README.md +++ b/python/packages/foundry_hosting/README.md @@ -6,6 +6,8 @@ Implementation is grouped by protocol: `_responses_agent.py` and `_responses_wor and `_responses_shared.py` contains their wire and streaming helpers. Invocations uses the same `_invocations_agent.py`, `_invocations_workflow.py`, and `_invocations_shared.py` layout. Cross-protocol request contracts, option validation, and agent/workflow factories live in `_shared.py`. Applications use the public package imports below. +Both Invocations hosts inherit directly from the SDK host; shared functions handle identity, wire conversion, +and streaming cleanup without a combined agent/workflow host. ## Choose the host @@ -49,11 +51,73 @@ returning a new wrapper around shared executors does not isolate execution state Workflow hosts expose checkpoint storage, not agent-session storage or agent history/background policies. Responses workflows retain `prepare_options`; Invocations workflows use the parser's -explicit workflow kwargs instead. Existing SDK options and agent approval-store APIs are unchanged. +explicit workflow kwargs instead. Agent approval-store APIs are unchanged. Agent hosts reject `WorkflowAgent`, whether supplied directly or returned by a factory. `checkpoint_store_provider` is available only on workflow hosts. No stored state is converted between native workflow and agent execution models. +## Explicit host configuration + +Responses hosts construct the upstream +[`ResponsesServerOptions`](https://pypi.org/project/azure-ai-agentserver-responses/) internally. +Use first-class host parameters rather than importing an SDK options bag: + +| Parameter | Available on | Meaning and SDK consumer | +| --- | --- | --- | +| `log_level=None` | All four hosts | A literal `"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`, or `"CRITICAL"`, forwarded unchanged to the AgentServer SDK constructor. `None` preserves the SDK default. | +| `max_history_items: int \| None = None` | Responses agent host, with `history_source="host_managed"` | A positive cap on prior outer history items, passed to the SDK history loader as `default_fetch_history_count`. `None` supplies no override and inherits the installed supported SDK's default. | +| `sse_keep_alive_interval_seconds: int \| None = None` | All four hosts with framed SSE | A positive streaming heartbeat interval in seconds. Responses forwards it to SDK streaming routes; Invocations sets the host's SDK heartbeat configuration. `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. | +| `shutdown_grace_period_seconds: int = 10` | Both Responses hosts | A positive shutdown wait in seconds. The SDK also uses it when DELETE waits for response persistence. It is not a request or inference timeout. | +| `resilient_background: bool = False` | Both Responses hosts | Uses the SDK's recovery setting without renaming it. It permits eligible recovery, not background execution or exactly-once side effects. | + +`max_history_items=None` does **not** promise unlimited history or map to SDK `-1`. +Supported SDK versions can have different defaults. Explicit caps count prior history items, not tokens, +turns, retention, or workflow checkpoints. Other agent history modes and workflow hosts do not accept a cap. + +Explicit heartbeat intervals must be positive: zero, negative values, and booleans are invalid, not +force-disable switches or requests to inherit the environment. The SDK still parses its environment +configuration during construction; a malformed `SSE_KEEPALIVE_INTERVAL` can fail even with an explicit interval. +The agent-only `legacy_wire_format=True` disables heartbeats and cannot be combined with an explicit interval. + +For regular-agent recovery, select both `history_source="inference_service"` and +`background_source="inference_service"`. Native workflows recover from their own checkpoints. +The SDK recovery setting itself is provider-independent; the inference-service requirement is a limitation +of the regular-agent adapter, which currently resumes only saved provider continuations. +Follow the SDK's task-subsystem and persistent-store requirements. Neither outer workflow background execution nor +recovery permission enables provider background on constituent agents. + +Host `default_model`, `create_span_hook`, `additional_server_version`, and `steerable_conversations` +are not first-class parameters. Framework-generated SDK options leave model defaults, span callbacks, and +custom version stamping unset, and keep steering disabled. Framework observability is unchanged. +Configure inference defaults on the agent or client. The former SDK `default_model` could fill the outer +request model and thereby override inference; it did not select a hosted agent. Removing that host fallback +does not change caller-supplied request models, option-hook timing, option precedence, or routing. + +### SDK constructor arguments + +All four hosts accept `agent_server_kwargs: Mapping[str, Any] | None = None`. +This caller-owned mapping is expanded only into the corresponding AgentServer SDK constructor. +Hosting does not inspect, validate, filter, or mutate its contents. Python and the SDK report invalid +arguments, including collisions with arguments the framework already supplies; there is no custom merge policy. +An SDK argument reaching this boundary does not mean every adapter path supports its behavior. + +Move SDK-specific top-level arguments into this mapping. For an application-owned `agent` and `asyncapi_spec`: + +```python +from agent_framework_foundry_hosting import InvocationsHostServer + +# Before: InvocationsHostServer(agent=agent, asyncapi_spec_json=asyncapi_spec) +server = InvocationsHostServer( + agent=agent, + agent_server_kwargs={"asyncapi_spec_json": asyncapi_spec}, +) +``` + +The constructors no longer accept arbitrary `**kwargs`. Removed top-level names, including `options=` +and old parser/storage aliases, raise Python `TypeError` before construction, even when their value is `None`. +Use the first-class parameters above for framework-supported configuration. Runtime model options, +`client_kwargs`, and `function_invocation_kwargs` keep their existing meanings and validation. + ## Toolbox tool approvals `FoundryToolbox` uses each discovered tool's `_meta.tool_configuration.require_approval` when @@ -200,8 +264,9 @@ history or decode pending replies. `agent=workflow.as_agent()` is no longer supp Wrapper context providers, history, event projection, and request-info translation are not silently unwrapped. See [Migrate `WorkflowAgent` hosting](#migrate-workflowagent-hosting). -With `resilient_background=True`, the application must also enable the -AgentServer resilient task subsystem. Host-owned checkpoint/output pairs +With `resilient_background=True`, native workflows recover through the +AgentServer resilient task subsystem; SDK-specific task initialization and persistence requirements still apply. +Host-owned checkpoint/output pairs recover emitted output and usage without selecting newer unpaired workflow state. This is not an exactly-once guarantee for external tools: make side-effecting operations idempotent. Legacy unscoped workflow state is not @@ -291,7 +356,7 @@ Outer background work always uses the caller-visible `response.id` for polling; background automatically. `background_source="host_managed"` (default) uses only the outer background worker. `background_source="inference_service"` is a separate opt-in for `history_source="inference_service"` with a storing Responses client. Its private continuation token is saved under the outer ID and never returned to the caller. -Use `ResponsesServerOptions(resilient_background=True)` to permit recovery from a **saved** token; a crash before +Use `resilient_background=True` to permit recovery from a **saved** token; a crash before the token is saved cannot safely restart the inner job. Completed polling output, including local function calls, results, and usage, is saved together with the next token in the private response-ID snapshot before emission. Outer output checkpoints record which saved batches have been emitted, so recovery restores their usage and @@ -304,13 +369,14 @@ another background response and saves its next token. A crash after a local tool next token is saved can still repeat the tool on recovery; use idempotent tools or avoid provider background for side-effecting local tools. This mechanism does not provide exactly-once tool execution. Provider background and steering cannot be combined. -Regular agent runs without this opt-in are not crash-replayable. **Steering is temporarily unavailable:** -`steerable_conversations=True` fails during host construction, before enabling the process-wide TaskManager. +Regular agent runs without this opt-in are not crash-replayable. **Steering is unavailable:** +`steerable_conversations` and the upstream `options` bag are not host parameters; framework-generated SDK +options keep steering disabled. The current AgentServer SDK retains unbounded futures for rejected turns when its steering queue fills. Do not -use a queue-length precheck: another worker can append before it. The guard can be removed only after -[Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233) ships in an -official `azure-ai-agentserver-core` wheel, the minimum dependency and `uv.lock` are updated, and a -concurrent queue-overflow regression proves rejected turns leave no pending futures. No future SDK +use a queue-length precheck: another worker can append before it. Future steering support requires an +official `azure-ai-agentserver-core` wheel containing +[Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233), updated minimum +dependencies and `uv.lock`, and a concurrent queue-overflow regression proving rejected turns leave no pending futures. No future SDK version is assumed. Non-steerable agent background polling is unchanged. Native CreateResponse generation fields become MAF runtime options (notably `max_output_tokens` -> `max_tokens` and diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py index 5a55661ac97..c1430210ca4 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py @@ -4,21 +4,74 @@ from __future__ import annotations -from collections.abc import Awaitable, Callable -from typing import Any +import asyncio +import inspect +import json +import logging +import sys +import warnings +import weakref +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping +from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager +from contextvars import Token +from copy import deepcopy +from typing import Any, Literal, cast -from agent_framework import SessionStore +from agent_framework import AgentSession, ResponseStream, SessionStore, SupportsAgentRun +from agent_framework._telemetry import mark_feature_used +from azure.ai.agentserver.core import ( + FoundryAgentRequestContext, + get_request_context, + reset_request_context, + set_request_context, +) +from azure.ai.agentserver.invocations import InvocationAgentServerHost from starlette.requests import Request +from starlette.responses import JSONResponse, Response +from ._feature_usage import FeatureIndex from ._invocations_shared import ( InvocationOptionsHook, - _InvocationsHostServer, # pyright: ignore[reportPrivateUsage] + _invocation_failure, # pyright: ignore[reportPrivateUsage] + _InvocationStreamingResponse, # pyright: ignore[reportPrivateUsage] + _sse, # pyright: ignore[reportPrivateUsage] + invocation_partition_key, + invocation_request_scope, +) +from ._scope import FoundryRequestScope +from ._shared import ( + AgentSource, + InvocationRun, + UnsupportedOptions, + is_agent, + resolve_agent, + validate_agent_source, + validate_request_options, + validate_sse_keep_alive_interval, + validate_unsupported_options, ) -from ._shared import AgentSource, InvocationRun, UnsupportedOptions -from ._state_store import StoreProvider +from ._state_store import AgentSessionStoreProvider, StoreProvider + +logger = logging.getLogger(__name__) + +_AGENT_CONTROLLED_FIELDS = frozenset({ + "additional_function_arguments", + "client_kwargs", + "compaction_strategy", + "function_invocation_kwargs", + "instructions", + "middleware", + "session", + "tokenizer", + "tools", +}) -class InvocationsHostServer(_InvocationsHostServer): +class _UnsupportedAgentOptions(TypeError): + """The agent cannot accept the caller's run options under the selected policy.""" + + +class InvocationsHostServer(InvocationAgentServerHost): """Host an agent with application-defined Invocations turns.""" def __init__( @@ -31,7 +84,9 @@ def __init__( prepare_options: InvocationOptionsHook | None = None, unsupported_options: UnsupportedOptions = "warn", legacy_wire_format: bool = False, - **kwargs: Any, + sse_keep_alive_interval_seconds: int | None = None, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, ) -> None: """Create an agent host with session-backed application turns. @@ -54,20 +109,275 @@ def __init__( options. Return the replacement mapping; hosting validates it before agent dispatch. unsupported_options: Policy for agents without runtime options: `"warn"`, `"ignore"`, or `"error"`. legacy_wire_format: Deprecated agent-only plain-text and raw-stream compatibility format. - **kwargs: Additional AgentServer host settings forwarded to the SDK. + Heartbeats are disabled in this format. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + An explicit interval cannot be combined with `legacy_wire_format=True`. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. Raises: - TypeError: If the agent source or parser/hook is invalid, or native workflow settings are supplied. + TypeError: If the agent source, parser/hook, heartbeat type, or legacy-format flag is invalid. + ValueError: If the heartbeat interval is nonpositive, raw legacy output requests heartbeats, + or the unsupported-options policy is invalid. """ - if {"workflow", "checkpoint_store_provider"}.intersection(kwargs): - raise TypeError("Use InvocationsWorkflowHostServer for native workflows and checkpoint storage.") + validate_agent_source(agent) + if parse_input is not None and not callable(parse_input): + raise TypeError("parse_input must be a callable.") + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + if not isinstance(legacy_wire_format, bool): + raise TypeError("legacy_wire_format must be a boolean.") + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + if legacy_wire_format and sse_keep_alive_interval_seconds is not None: + raise ValueError("SSE heartbeats require framed SSE; disable legacy_wire_format.") + unsupported_options = validate_unsupported_options(unsupported_options) super().__init__( - agent=agent, openapi_spec=openapi_spec, - agent_session_store_provider=agent_session_store_provider, - parse_input=parse_input, - prepare_options=prepare_options, - unsupported_options=unsupported_options, - legacy_wire_format=legacy_wire_format, - **kwargs, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) + if legacy_wire_format: + self.config.sse_keepalive_interval = 0 + elif sse_keep_alive_interval_seconds is not None: + self.config.sse_keepalive_interval = sse_keep_alive_interval_seconds + self._agent = agent + self._owns_request_agent = not is_agent(agent) + self._parse_input = parse_input + self._prepare_options = prepare_options + self._unsupported_options = unsupported_options + self._legacy_wire_format = legacy_wire_format + self._session_store_provider = ( + AgentSessionStoreProvider(store_name="invocation_sessions") + if agent_session_store_provider is None + else agent_session_store_provider + ) + self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( + weakref.WeakValueDictionary() + ) + if legacy_wire_format: + message = ( + "legacy_wire_format=True is deprecated; migrate Invocations clients to JSON responses " + "and framed SSE events before removing this compatibility mode." + ) + warnings.warn(message, DeprecationWarning, stacklevel=2) + logger.warning("DEPRECATION: %s", message) + self.invoke_handler(self._handle_invoke) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + @asynccontextmanager + async def _request_agent(self, request: Request) -> AsyncGenerator[SupportsAgentRun]: + agent = await resolve_agent(self._agent, request) + async with AsyncExitStack() as resources: + if self._owns_request_agent and isinstance(agent, AbstractAsyncContextManager): + await resources.enter_async_context(agent) + yield agent + + @asynccontextmanager + async def _request_session( + self, + partition_key: str | tuple[str, str], + context: FoundryAgentRequestContext, + *, + hosted_scope: FoundryRequestScope | None = None, + ) -> AsyncGenerator[AgentSession]: + session_id = ( + json.dumps(partition_key, separators=(",", ":")) if isinstance(partition_key, tuple) else partition_key ) + try: + provider = self._session_store_provider + if hosted_scope is not None and not self.config.session_id and type(provider) is AgentSessionStoreProvider: + store = provider._get_scoped_store(context, hosted_scope) # pyright: ignore[reportPrivateUsage] + else: + store = provider.get_store(config=self.config, platform_context=context) + session = await store.get(session_id) + if session is None: + session = AgentSession(session_id=session_id) + except Exception: + logger.exception("Failed to load invocation session") + raise + try: + yield session + finally: + failure = sys.exc_info()[1] + try: + await store.set(session_id, session) + except Exception as exc: + logger.exception("Failed to persist invocation session") + if failure is None: + raise + if isinstance(failure, Exception): + raise RuntimeError( + f"Invocation failed: {str(failure) or type(failure).__name__}; " + f"session persistence also failed: {str(exc) or type(exc).__name__}" + ) from exc + + async def _parse(self, request: Request) -> InvocationRun: + if self._parse_input is not None: + result = self._parse_input(request) + parsed = await result if inspect.isawaitable(result) else result + if not isinstance(parsed, InvocationRun): + raise TypeError("parse_input must return InvocationRun.") + return parsed + + payload = await request.json() + if not isinstance(payload, dict): + raise ValueError("The invocation must be a JSON object.") + body = cast(Mapping[str, Any], payload) + message = body.get("message") + stream = body.get("stream", False) + options = body.get("options", {}) + if not isinstance(message, str): + raise ValueError("message must be a string.") + if not isinstance(stream, bool): + raise ValueError("stream must be a boolean.") + if not isinstance(options, dict): + raise ValueError("options must be an object.") + return InvocationRun( + messages=message if stream else [message], options=cast(Mapping[str, Any], options), stream=stream + ) + + async def _options(self, request: Request, parsed: InvocationRun) -> dict[str, Any]: + options = deepcopy(dict(parsed.options)) + if self._prepare_options is not None: + result = self._prepare_options(request, options) + if inspect.isawaitable(result): + result = await result + if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): + raise TypeError("prepare_options must return a mapping of MAF run options with string keys.") + options = deepcopy(dict(result)) + validate_request_options(options) + reserved = _AGENT_CONTROLLED_FIELDS.intersection(options) + if reserved: + raise ValueError(f"Invocations options cannot set agent-controlled fields: {', '.join(sorted(reserved))}.") + return options + + def _agent_kwargs(self, agent: SupportsAgentRun, options: dict[str, Any]) -> dict[str, Any]: + if not options: + return {} + try: + inspect.signature(agent.run).bind_partial(options=options) + except (TypeError, ValueError): + if self._unsupported_options == "error": + raise _UnsupportedAgentOptions("The hosted agent does not accept caller runtime options.") from None + if self._unsupported_options == "warn": + logger.warning("Agent doesn't support runtime options. They will be ignored.") + return {} + return {"options": options} + + @staticmethod + async def _close_interrupted_stream(stream: object) -> None: + if isinstance(stream, ResponseStream): + close = getattr(cast(object, stream), "close", None) + if close is None: + logger.warning("The installed core cannot close an interrupted agent stream.") + return + else: + close = getattr(stream, "aclose", None) + if close is not None: + await close() + + async def _handle_invoke(self, request: Request) -> Response: + """Invoke the configured agent with a trusted request.""" + context = get_request_context() + try: + hosted_scope = invocation_request_scope(self.config, request, context) + partition_key = invocation_partition_key(self.config, context=context, scope=hosted_scope) + except RuntimeError as exc: + logger.error("Failed to resolve Invocations session: %s", exc) + return JSONResponse({"error": str(exc)}, status_code=500) + + try: + parsed = await self._parse(request) + options = await self._options(request, parsed) + except (TypeError, ValueError) as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception: + logger.exception("Failed to prepare Invocations request") + return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) + + if parsed.stream: + if options and self._unsupported_options == "error" and is_agent(self._agent): + try: + self._agent_kwargs(self._agent, options) + except _UnsupportedAgentOptions as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + + async def stream_response() -> AsyncGenerator[str]: + token: Token[FoundryAgentRequestContext] | None = set_request_context(context) + try: + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock, self._request_agent(request) as agent: + run_kwargs = self._agent_kwargs(agent, options) + async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: + stream = agent.run(parsed.messages, session=session, stream=True, **run_kwargs) + completed = False + try: + async for update in stream: + if update.text: + frame = ( + update.text + if self._legacy_wire_format + else _sse("delta", {"text": update.text}) + ) + # The SDK may close a suspended generator from another task. + reset_request_context(token) + token = None + yield frame + token = set_request_context(context) + if isinstance(stream, ResponseStream): + await stream.get_final_response() + completed = True + finally: + if not completed: + if token is None: + token = set_request_context(context) + try: + await self._close_interrupted_stream(stream) + except Exception: + logger.exception("Failed to close interrupted Invocations agent stream") + if not self._legacy_wire_format: + session_id = partition_key[0] if isinstance(partition_key, tuple) else partition_key + if token is not None: + reset_request_context(token) + token = None + yield _sse("done", {"session_id": session_id}) + except _UnsupportedAgentOptions as exc: + if self._legacy_wire_format: + raise + if token is not None: + reset_request_context(token) + token = None + yield _sse("error", {"message": str(exc), "code": "unsupported_options", "status": 400}) + except Exception as exc: + logger.exception("Invocations agent stream failed") + if self._legacy_wire_format: + raise + message, status, code = _invocation_failure(exc) + if token is not None: + reset_request_context(token) + token = None + yield _sse("error", {"message": message, "code": code, "status": status}) + finally: + if token is not None: + reset_request_context(token) + + return _InvocationStreamingResponse(stream_response()) + + try: + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock, self._request_agent(request) as agent: + run_kwargs = self._agent_kwargs(agent, options) + async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: + response = await agent.run(parsed.messages, session=session, **run_kwargs) + if self._legacy_wire_format: + return Response(content=response.text) + return JSONResponse({"response": response.text}) + except _UnsupportedAgentOptions as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception as exc: + logger.exception("Invocations agent request failed") + message, status, _ = _invocation_failure(exc) + return JSONResponse({"error": message}, status_code=status) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py index 98f5a5de3d7..4a85faa16d1 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py @@ -1,94 +1,87 @@ # Copyright (c) Microsoft. All rights reserved. -"""Share Invocations protocol, state, and execution plumbing between agent and workflow hosts.""" +"""Share Invocations identity, wire conversion, and streaming cleanup helpers.""" from __future__ import annotations -import asyncio -import inspect import json -import logging import math -import sys -import uuid -import warnings -import weakref from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping, Sequence -from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager -from contextvars import Token -from copy import deepcopy from dataclasses import asdict, is_dataclass from typing import Any, cast -from agent_framework import ( - AgentResponse, - AgentResponseUpdate, - AgentSession, - ChatResponse, - ChatResponseUpdate, - CheckpointStorage, - ResponseStream, - SessionStore, - SupportsAgentRun, - WorkflowCheckpointException, - WorkflowEvent, -) -from agent_framework._telemetry import mark_feature_used +from agent_framework import AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate, WorkflowEvent from anyio import CancelScope -from azure.ai.agentserver.core import ( - FoundryAgentRequestContext, - get_request_context, - reset_request_context, - set_request_context, -) +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext, get_request_context from azure.ai.agentserver.core.storage import FoundryStorageConflictError, FoundryStoragePreconditionError -from azure.ai.agentserver.invocations import InvocationAgentServerHost from pydantic import BaseModel from starlette.requests import Request -from starlette.responses import JSONResponse, Response, StreamingResponse +from starlette.responses import StreamingResponse from starlette.types import Receive, Scope, Send -from ._feature_usage import FeatureIndex from ._scope import FoundryRequestScope -from ._shared import ( - AgentSource, - InvocationRun, - UnsupportedOptions, - WorkflowResolver, - WorkflowSource, - WorkflowTurn, - is_agent, - resolve_agent, - validate_agent_source, - validate_request_options, - validate_unsupported_options, - workflow_agents, -) -from ._state_store import AgentSessionStoreProvider, CheckpointStoreProvider, ContextScopedStoreProvider, StoreProvider -from ._workflow_state import HostedWorkflowRun, WorkflowBlockedError, WorkflowConflictError - -logger = logging.getLogger(__name__) - -InvocationParser = Callable[[Request], InvocationRun | WorkflowTurn[Any] | Awaitable[InvocationRun | WorkflowTurn[Any]]] +from ._workflow_state import WorkflowBlockedError, WorkflowConflictError + InvocationOptionsHook = Callable[[Request, dict[str, Any]], Mapping[str, Any] | Awaitable[Mapping[str, Any]]] -_AGENT_CONTROLLED_FIELDS = frozenset({ - "additional_function_arguments", - "client_kwargs", - "compaction_strategy", - "function_invocation_kwargs", - "instructions", - "middleware", - "session", - "tokenizer", - "tools", -}) -_MAX_WORKFLOW_SNAPSHOT_EVENTS = 4096 -_MAX_WORKFLOW_SNAPSHOT_BYTES = 4 * 1024 * 1024 + +def invocation_partition_key( + config: AgentConfig, + *, + context: FoundryAgentRequestContext | None = None, + scope: FoundryRequestScope | None = None, +) -> str | tuple[str, str]: + """Resolve the session key, preserving hosted session/user boundaries.""" + if context is None: + context = get_request_context() + + if config.is_hosted: + if not context.user_id: + raise RuntimeError( + "The hosted environment is missing user_id in the request context. " + "Please ensure that the request is coming from a valid Foundry platform service." + ) + hosted_scope = scope or FoundryRequestScope.from_context(config, context) + return hosted_scope.session_id, context.user_id + + if not context.session_id: + raise RuntimeError( + "The request context is missing session_id. Please ensure that the request is a valid request." + ) + return context.session_id -class _UnsupportedAgentOptions(TypeError): - """The agent cannot accept the caller's run options under the selected policy.""" +def invocation_request_scope( + config: AgentConfig, request: Request, context: FoundryAgentRequestContext +) -> FoundryRequestScope | None: + """Resolve hosted identity, requiring a routed query when the platform session is unconfigured.""" + if not config.is_hosted: + return None + if not context.user_id: + raise RuntimeError("The hosted environment is missing user_id in the request context.") + + routed_session_ids = request.query_params.getlist("agent_session_id") + if len(routed_session_ids) > 1: + raise RuntimeError("Hosted Invocations requires exactly one agent_session_id query parameter.") + routed_session_id = routed_session_ids[0] if routed_session_ids else None + if config.session_id: + if routed_session_id is not None and routed_session_id != config.session_id: + raise RuntimeError("The request agent_session_id does not match the platform session ID.") + return FoundryRequestScope.from_context(config, context) + + if not routed_session_id or routed_session_id != context.session_id: + raise RuntimeError( + "Hosted Invocations without FOUNDRY_AGENT_SESSION_ID require an explicit routed " + "agent_session_id query parameter matching the request context." + ) + if not context.call_id: + raise RuntimeError("Foundry hosted requests require a trusted user ID and call ID.") + return FoundryRequestScope( + session_id=routed_session_id, + user_id=context.user_id, + call_id=context.call_id, + is_hosted=True, + ) class _WorkflowOutputError(TypeError): @@ -202,7 +195,9 @@ def _workflow_output(value: Any) -> Any: raise _WorkflowOutputError(f"Workflow output type {type(value).__name__} is not JSON-serializable.") from exc -def _workflow_frame(event: WorkflowEvent[Any]) -> dict[str, Any] | None: +def _workflow_frame( # pyright: ignore[reportUnusedFunction] + event: WorkflowEvent[Any], +) -> dict[str, Any] | None: if event.type not in ("output", "request_info"): return None frame: dict[str, Any] = {"type": event.type, "data": event.data} @@ -214,7 +209,9 @@ def _workflow_frame(event: WorkflowEvent[Any]) -> dict[str, Any] | None: return cast(dict[str, Any], _workflow_output(frame)) -def _workflow_snapshot_frames(snapshot: Mapping[str, Any] | None) -> list[dict[str, Any]]: +def _workflow_snapshot_frames( # pyright: ignore[reportUnusedFunction] + snapshot: Mapping[str, Any] | None, +) -> list[dict[str, Any]]: if snapshot is None or set(snapshot) != {"output"} or not isinstance(snapshot["output"], list): raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") frames: list[dict[str, Any]] = [] @@ -234,11 +231,15 @@ def _workflow_snapshot_frames(snapshot: Mapping[str, Any] | None) -> list[dict[s return frames -def _workflow_frame_size(frame: Mapping[str, Any]) -> int: +def _workflow_frame_size( # pyright: ignore[reportUnusedFunction] + frame: Mapping[str, Any], +) -> int: return len(json.dumps(frame, ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) -def _sse(event: str, data: Mapping[str, Any]) -> str: +def _sse( # pyright: ignore[reportUnusedFunction] + event: str, data: Mapping[str, Any] +) -> str: return f"event: {event}\ndata: {json.dumps(data)}\n\n" @@ -251,7 +252,9 @@ def _invocation_failure(exc: Exception) -> tuple[str, int, str]: return "Agent invocation failed.", 500, "invocation_failed" -def _workflow_failure(exc: Exception) -> tuple[str, int, str]: +def _workflow_failure( # pyright: ignore[reportUnusedFunction] + exc: Exception, +) -> tuple[str, int, str]: if isinstance(exc, _WorkflowOutputError): return str(exc), 500, "invalid_workflow_output" cause: BaseException | None = exc @@ -297,7 +300,7 @@ async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: await self._content.aclose() -class _WorkflowStreamingResponse(_InvocationStreamingResponse): +class _WorkflowStreamingResponse(_InvocationStreamingResponse): # pyright: ignore[reportUnusedClass] def __init__(self, content: AsyncGenerator[str], *, keep_alive: bool) -> None: super().__init__(content) self._keep_alive = keep_alive @@ -318,539 +321,3 @@ async def stream_response(self, send: Send) -> None: # Without the SDK pump, finish its wrapper in the task that created its context tokens. async for _ in self.body_iterator: pass - - -class _InvocationsHostServer(InvocationAgentServerHost): # pyright: ignore[reportUnusedClass] - """Share protocol, scoped state, and execution plumbing between the public hosts.""" - - def __init__( - self, - agent: AgentSource[Request] | None = None, - *, - workflow: WorkflowSource[Request] | None = None, - openapi_spec: dict[str, Any] | None = None, - agent_session_store_provider: StoreProvider[SessionStore] | None = None, - checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - parse_input: InvocationParser | None = None, - prepare_options: InvocationOptionsHook | None = None, - unsupported_options: UnsupportedOptions = "warn", - legacy_wire_format: bool = False, - **kwargs: Any, - ) -> None: - """Initialize shared protocol plumbing after the public host selects its execution model.""" - if "parse_request" in kwargs: - raise TypeError("parse_request was renamed to parse_input.") - if agent is not None and workflow is not None: - raise TypeError("Provide exactly one of agent or workflow.") - if agent is None and workflow is None: - raise TypeError( - "Provide exactly one of agent or workflow; " - "agent must be an agent instance or a callable that creates one." - ) - if workflow is None: - validate_agent_source(agent) - elif parse_input is None: - raise TypeError("parse_input is required when hosting a native Workflow.") - if parse_input is not None and not callable(parse_input): - raise TypeError("parse_input must be a callable.") - if prepare_options is not None and not callable(prepare_options): - raise TypeError("prepare_options must be a callable.") - if not isinstance(legacy_wire_format, bool): - raise TypeError("legacy_wire_format must be a boolean.") - if workflow is not None: - if legacy_wire_format: - raise ValueError("legacy_wire_format=True is incompatible with native workflows.") - if prepare_options is not None: - raise ValueError("prepare_options is agent-only; use the workflow parser's explicit client_kwargs.") - if agent_session_store_provider is not None: - raise ValueError( - "agent_session_store_provider is agent-only; use checkpoint_store_provider for workflows." - ) - self._workflow_resolver: WorkflowResolver[Request] | None = ( - WorkflowResolver(workflow) if workflow is not None else None - ) - super().__init__(openapi_spec=openapi_spec, **kwargs) - self._agent = agent - self._owns_request_agent = agent is not None and not is_agent(agent) - self._parse_input = parse_input - self._prepare_options = prepare_options - self._unsupported_options = validate_unsupported_options(unsupported_options) - self._legacy_wire_format = legacy_wire_format - self._session_store_provider = ( - AgentSessionStoreProvider(store_name="invocation_sessions") - if agent_session_store_provider is None - else agent_session_store_provider - ) - self._checkpoint_storage_provider = ( - CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider - ) - self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( - weakref.WeakValueDictionary() - ) - if legacy_wire_format: - message = ( - "legacy_wire_format=True is deprecated; migrate Invocations clients to JSON responses " - "and framed SSE events before removing this compatibility mode." - ) - warnings.warn(message, DeprecationWarning, stacklevel=3) - logger.warning("DEPRECATION: %s", message) - self.invoke_handler(self._handle_invoke) - mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) - - def _partition_key( - self, - *, - context: FoundryAgentRequestContext | None = None, - scope: FoundryRequestScope | None = None, - ) -> str | tuple[str, str]: - """Get the partition key for the current request. - - A hosted partition key is a tuple containing the session ID and user ID, - preserving their boundaries. Locally, the key is just the session ID. In the - Foundry hosted environment, the partition key is used to maintain isolation between - different sessions and users, such that one user cannot access another user's sessions. - - Returns: - The partition key for the current request. - - Exceptions: - RuntimeError: If the context doesn't contain the expected IDs. - """ - if context is None: - context = get_request_context() - - if self.config.is_hosted: - if not context.user_id: - raise RuntimeError( - "The hosted environment is missing user_id in the request context. " - "Please ensure that the request is coming from a valid Foundry platform service." - ) - hosted_scope = scope or FoundryRequestScope.from_context(self.config, context) - return hosted_scope.session_id, context.user_id - - if not context.session_id: - raise RuntimeError( - "The request context is missing session_id. Please ensure that the request is a valid request." - ) - - return context.session_id - - def _hosted_scope(self, request: Request, context: FoundryAgentRequestContext) -> FoundryRequestScope: - """Accept an unconfigured session only when the routed Invocations query identifies it.""" - if not context.user_id: - raise RuntimeError("The hosted environment is missing user_id in the request context.") - - routed_session_ids = request.query_params.getlist("agent_session_id") - if len(routed_session_ids) > 1: - raise RuntimeError("Hosted Invocations requires exactly one agent_session_id query parameter.") - routed_session_id = routed_session_ids[0] if routed_session_ids else None - if self.config.session_id: - if routed_session_id is not None and routed_session_id != self.config.session_id: - raise RuntimeError("The request agent_session_id does not match the platform session ID.") - return FoundryRequestScope.from_context(self.config, context) - - if not routed_session_id or routed_session_id != context.session_id: - raise RuntimeError( - "Hosted Invocations without FOUNDRY_AGENT_SESSION_ID require an explicit routed " - "agent_session_id query parameter matching the request context." - ) - if not context.call_id: - raise RuntimeError("Foundry hosted requests require a trusted user ID and call ID.") - return FoundryRequestScope( - session_id=routed_session_id, - user_id=context.user_id, - call_id=context.call_id, - is_hosted=True, - ) - - @asynccontextmanager - async def _request_agent(self, request: Request) -> AsyncGenerator[SupportsAgentRun]: - if self._agent is None: - raise RuntimeError("No agent is configured for Invocations.") - agent = await resolve_agent(self._agent, request) - async with AsyncExitStack() as resources: - if self._owns_request_agent and isinstance(agent, AbstractAsyncContextManager): - await resources.enter_async_context(agent) - yield agent - - @asynccontextmanager - async def _request_session( - self, - partition_key: str | tuple[str, str], - context: FoundryAgentRequestContext, - *, - hosted_scope: FoundryRequestScope | None = None, - ) -> AsyncGenerator[AgentSession]: - session_id = ( - json.dumps(partition_key, separators=(",", ":")) if isinstance(partition_key, tuple) else partition_key - ) - try: - provider = self._session_store_provider - if hosted_scope is not None and not self.config.session_id and type(provider) is AgentSessionStoreProvider: - store = provider._get_scoped_store(context, hosted_scope) # pyright: ignore[reportPrivateUsage] - else: - store = provider.get_store(config=self.config, platform_context=context) - session = await store.get(session_id) - if session is None: - session = AgentSession(session_id=session_id) - except Exception: - logger.exception("Failed to load invocation session") - raise - try: - yield session - finally: - failure = sys.exc_info()[1] - try: - await store.set(session_id, session) - except Exception as exc: - logger.exception("Failed to persist invocation session") - if failure is None: - raise - if isinstance(failure, Exception): - raise RuntimeError( - f"Invocation failed: {str(failure) or type(failure).__name__}; " - f"session persistence also failed: {str(exc) or type(exc).__name__}" - ) from exc - - async def _parse(self, request: Request) -> InvocationRun | WorkflowTurn[Any]: - if self._parse_input is not None: - result = self._parse_input(request) - parsed = await result if inspect.isawaitable(result) else result - if self._workflow_resolver is not None: - if not isinstance(parsed, WorkflowTurn): - raise TypeError("parse_input must return WorkflowTurn.") - elif not isinstance(parsed, InvocationRun): - raise TypeError("parse_input must return InvocationRun.") - return parsed - - payload = await request.json() - if not isinstance(payload, dict): - raise ValueError("The invocation must be a JSON object.") - body = cast(Mapping[str, Any], payload) - message = body.get("message") - stream = body.get("stream", False) - options = body.get("options", {}) - if not isinstance(message, str): - raise ValueError("message must be a string.") - if not isinstance(stream, bool): - raise ValueError("stream must be a boolean.") - if not isinstance(options, dict): - raise ValueError("options must be an object.") - return InvocationRun( - messages=message if stream else [message], options=cast(Mapping[str, Any], options), stream=stream - ) - - async def _options(self, request: Request, parsed: InvocationRun) -> dict[str, Any]: - options = deepcopy(dict(parsed.options)) - if self._prepare_options is not None: - result = self._prepare_options(request, options) - if inspect.isawaitable(result): - result = await result - if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): - raise TypeError("prepare_options must return a mapping of MAF run options with string keys.") - options = deepcopy(dict(result)) - validate_request_options(options) - reserved = _AGENT_CONTROLLED_FIELDS.intersection(options) - if reserved: - raise ValueError(f"Invocations options cannot set agent-controlled fields: {', '.join(sorted(reserved))}.") - return options - - def _agent_kwargs(self, agent: SupportsAgentRun, options: dict[str, Any]) -> dict[str, Any]: - if not options: - return {} - try: - inspect.signature(agent.run).bind_partial(options=options) - except (TypeError, ValueError): - if self._unsupported_options == "error": - raise _UnsupportedAgentOptions("The hosted agent does not accept caller runtime options.") from None - if self._unsupported_options == "warn": - logger.warning("Agent doesn't support runtime options. They will be ignored.") - return {} - return {"options": options} - - @staticmethod - async def _close_interrupted_stream(stream: object) -> None: - if isinstance(stream, ResponseStream): - close = getattr(cast(object, stream), "close", None) - if close is None: - logger.warning("The installed core cannot close an interrupted agent stream.") - return - else: - close = getattr(stream, "aclose", None) - if close is not None: - await close() - - async def _handle_invoke(self, request: Request) -> Response: - """Invoke the configured agent or native workflow with a trusted request.""" - context = get_request_context() - try: - hosted_scope = self._hosted_scope(request, context) if self.config.is_hosted else None - partition_key = self._partition_key(context=context, scope=hosted_scope) - except RuntimeError as exc: - logger.error("Failed to resolve Invocations session: %s", exc) - return JSONResponse({"error": str(exc)}, status_code=500) - - try: - parsed = await self._parse(request) - options = await self._options(request, parsed) if isinstance(parsed, InvocationRun) else {} - except (TypeError, ValueError) as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - except Exception: - logger.exception("Failed to prepare Invocations request") - return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) - - if isinstance(parsed, WorkflowTurn): - scope = hosted_scope or FoundryRequestScope.from_context(self.config, context) - return await self._handle_workflow(request, parsed, scope, context, partition_key) - - if parsed.stream: - if options and self._unsupported_options == "error" and is_agent(self._agent): - try: - self._agent_kwargs(self._agent, options) - except _UnsupportedAgentOptions as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - - async def stream_response() -> AsyncGenerator[str]: - token: Token[FoundryAgentRequestContext] | None = set_request_context(context) - try: - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock, self._request_agent(request) as agent: - run_kwargs = self._agent_kwargs(agent, options) - async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: - stream = agent.run(parsed.messages, session=session, stream=True, **run_kwargs) - completed = False - try: - async for update in stream: - if update.text: - frame = ( - update.text - if self._legacy_wire_format - else _sse("delta", {"text": update.text}) - ) - # The SDK may close a suspended generator from another task. - reset_request_context(token) - token = None - yield frame - token = set_request_context(context) - if isinstance(stream, ResponseStream): - await stream.get_final_response() - completed = True - finally: - if not completed: - if token is None: - token = set_request_context(context) - try: - await self._close_interrupted_stream(stream) - except Exception: - logger.exception("Failed to close interrupted Invocations agent stream") - if not self._legacy_wire_format: - session_id = partition_key[0] if isinstance(partition_key, tuple) else partition_key - if token is not None: - reset_request_context(token) - token = None - yield _sse("done", {"session_id": session_id}) - except _UnsupportedAgentOptions as exc: - if self._legacy_wire_format: - raise - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": str(exc), "code": "unsupported_options", "status": 400}) - except Exception as exc: - logger.exception("Invocations agent stream failed") - if self._legacy_wire_format: - raise - message, status, code = _invocation_failure(exc) - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": message, "code": code, "status": status}) - finally: - if token is not None: - reset_request_context(token) - - return _InvocationStreamingResponse(stream_response()) - - try: - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock, self._request_agent(request) as agent: - run_kwargs = self._agent_kwargs(agent, options) - async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: - response = await agent.run(parsed.messages, session=session, **run_kwargs) - if self._legacy_wire_format: - return Response(content=response.text) - return JSONResponse({"response": response.text}) - except _UnsupportedAgentOptions as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - except Exception as exc: - logger.exception("Invocations agent request failed") - message, status, _ = _invocation_failure(exc) - return JSONResponse({"error": message}, status_code=status) - - async def _handle_workflow( - self, - request: Request, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - context: FoundryAgentRequestContext, - partition_key: str | tuple[str, str], - ) -> Response: - if turn.stream: - - async def stream_response() -> AsyncGenerator[str]: - token: Token[FoundryAgentRequestContext] | None = set_request_context(context) - events = self._workflow_events(request, turn, scope, context, partition_key) - try: - async for frame in events: - wire_frame = _sse(frame["type"], frame) - reset_request_context(token) - token = None - yield wire_frame - token = set_request_context(context) - reset_request_context(token) - token = None - yield _sse("done", {"session_id": scope.session_id}) - except Exception as exc: - logger.exception("Invocations workflow stream failed") - message, status, code = _workflow_failure(exc) - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": message, "code": code, "status": status}) - finally: - if token is None: - token = set_request_context(context) - try: - with CancelScope(shield=True): - await events.aclose() - finally: - reset_request_context(token) - - return _WorkflowStreamingResponse(stream_response(), keep_alive=self.config.sse_keepalive_interval > 0) - - try: - frames = [frame async for frame in self._workflow_events(request, turn, scope, context, partition_key)] - return JSONResponse({"output": frames}) - except Exception as exc: - logger.exception("Invocations workflow request failed") - message, status, _ = _workflow_failure(exc) - return JSONResponse({"error": message}, status_code=status) - - async def _workflow_events( - self, - request: Request, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - context: FoundryAgentRequestContext, - partition_key: str | tuple[str, str], - ) -> AsyncGenerator[dict[str, Any]]: - resolver = self._workflow_resolver - if resolver is None: - raise RuntimeError("No native workflow is configured for Invocations.") - frames: list[dict[str, Any]] = [] - snapshot_bytes = 0 - deferred: list[dict[str, Any]] = [] - defer = False - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock: - workflow = await resolver.resolve(request) - run = await HostedWorkflowRun.prepare( - workflow, - scope=scope, - response_id=context.call_id or uuid.uuid4().hex, - config=self.config, - platform_context=context, - checkpoint_store_provider=self._checkpoint_storage_provider, - lineage_id="invocations", - stored=True, - replay_completed=True, - fresh_factory=True, - reclaim_superseded=True, - ) - if run.has_completed_output: - for frame in _workflow_snapshot_frames(run.snapshot): - yield frame - return - try: - turn = run.validate_turn(turn) - except (TypeError, ValueError) as exc: - raise _WorkflowRequestError(str(exc)) from exc - if ( - run.stored - and turn.input is not None - and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) - ): - try: - self._checkpoint_storage_provider.validate_checkpoint_value(turn.input) - except WorkflowCheckpointException as exc: - raise _WorkflowRequestError( - "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " - 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' - ) from exc - if ( - run.stored - and turn.responses is not None - and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) - ): - try: - for reply in turn.responses.values(): - self._checkpoint_storage_provider.validate_checkpoint_value(reply) - except WorkflowCheckpointException as exc: - raise _WorkflowRequestError( - "Stored native workflow reply uses an application type that is not checkpoint-allowlisted. " - 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' - ) from exc - committed = False - resources = AsyncExitStack() - try: - await run.claim() - try: - entered: set[int] = set() - for agent in workflow_agents(workflow): - if id(agent) not in entered and isinstance(agent, AbstractAsyncContextManager): - entered.add(id(agent)) - await resources.enter_async_context(agent) - events = run.events(turn) - resources.push_async_callback(events.aclose) - async for event in events: - frame = _workflow_frame(event) - if frame is None: - continue - frame_size = _workflow_frame_size(frame) - if ( - len(frames) >= _MAX_WORKFLOW_SNAPSHOT_EVENTS - or snapshot_bytes + frame_size > _MAX_WORKFLOW_SNAPSHOT_BYTES - ): - raise _WorkflowOutputError( - "Workflow output exceeds the bounded Invocations snapshot limit." - ) - snapshot_bytes += frame_size - frames.append(frame) - defer = defer or event.type == "request_info" - if defer: - deferred.append(frame) - else: - yield frame - finally: - # Starlette cancellation must not interrupt core/resource cleanup. - with CancelScope(shield=True): - await resources.__aexit__(*sys.exc_info()) - await run.commit({"output": frames}) - committed = True - finally: - if not committed: - failure = sys.exc_info()[1] - try: - with CancelScope(shield=True): - await run.abort() - except Exception as exc: - logger.exception("Failed to persist interrupted Invocations workflow state") - if failure is None: - raise - if isinstance(failure, Exception): - raise RuntimeError( - "Workflow invocation failed and its interrupted state could not be persisted." - ) from exc - # Pending reply authority is durable, and a reply need not wait on this stream's lock. - for frame in deferred: - yield frame diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py index 3f956209228..460467131cc 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py @@ -4,18 +4,55 @@ from __future__ import annotations -from collections.abc import Awaitable, Callable -from typing import Any +import asyncio +import inspect +import logging +import sys +import uuid +import weakref +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping +from contextlib import AbstractAsyncContextManager, AsyncExitStack +from contextvars import Token +from typing import Any, Literal -from agent_framework import CheckpointStorage +from agent_framework import CheckpointStorage, WorkflowCheckpointException +from agent_framework._telemetry import mark_feature_used +from anyio import CancelScope +from azure.ai.agentserver.core import ( + FoundryAgentRequestContext, + get_request_context, + reset_request_context, + set_request_context, +) +from azure.ai.agentserver.invocations import InvocationAgentServerHost from starlette.requests import Request +from starlette.responses import JSONResponse, Response -from ._invocations_shared import _InvocationsHostServer # pyright: ignore[reportPrivateUsage] -from ._shared import WorkflowSource, WorkflowTurn -from ._state_store import ContextScopedStoreProvider +from ._feature_usage import FeatureIndex +from ._invocations_shared import ( + _sse, # pyright: ignore[reportPrivateUsage] + _workflow_failure, # pyright: ignore[reportPrivateUsage] + _workflow_frame, # pyright: ignore[reportPrivateUsage] + _workflow_frame_size, # pyright: ignore[reportPrivateUsage] + _workflow_snapshot_frames, # pyright: ignore[reportPrivateUsage] + _WorkflowOutputError, # pyright: ignore[reportPrivateUsage] + _WorkflowRequestError, # pyright: ignore[reportPrivateUsage] + _WorkflowStreamingResponse, # pyright: ignore[reportPrivateUsage] + invocation_partition_key, + invocation_request_scope, +) +from ._scope import FoundryRequestScope +from ._shared import WorkflowResolver, WorkflowSource, WorkflowTurn, validate_sse_keep_alive_interval, workflow_agents +from ._state_store import CheckpointStoreProvider, ContextScopedStoreProvider +from ._workflow_state import HostedWorkflowRun +logger = logging.getLogger(__name__) -class InvocationsWorkflowHostServer(_InvocationsHostServer): +_MAX_WORKFLOW_SNAPSHOT_EVENTS = 4096 +_MAX_WORKFLOW_SNAPSHOT_BYTES = 4 * 1024 * 1024 + + +class InvocationsWorkflowHostServer(InvocationAgentServerHost): """Host a native, checkpoint-owned workflow with application-defined Invocations turns.""" def __init__( @@ -25,7 +62,9 @@ def __init__( parse_input: Callable[[Request], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], openapi_spec: dict[str, Any] | None = None, checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - **kwargs: Any, + sse_keep_alive_interval_seconds: int | None = None, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, ) -> None: """Create a native workflow host with explicit parsing and checkpoint ownership. @@ -41,28 +80,218 @@ def __init__( openapi_spec: Application OpenAPI specification forwarded to AgentServer. checkpoint_store_provider: Scoped checkpoint storage. Defaults to `CheckpointStoreProvider`. Native workflow history and pending replies are checkpoint-owned, not agent-session-owned. - **kwargs: Additional AgentServer host settings forwarded to the SDK. Agent option hooks and - legacy wire format are not supported. Output uses JSON or framed SSE workflow events. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. Raises: - TypeError: If the workflow is not a request-aware factory, the parser is invalid, or an agent-only - setting is supplied. + TypeError: If the workflow is not a request-aware factory, the parser is invalid, + or the heartbeat type is invalid. + ValueError: If the heartbeat interval is nonpositive. """ - forbidden = { - "agent", - "agent_session_store_provider", - "prepare_options", - "unsupported_options", - "legacy_wire_format", - }.intersection(kwargs) - if forbidden: - raise TypeError(f"Unsupported native workflow host settings: {', '.join(sorted(forbidden))}.") + self._workflow_resolver = WorkflowResolver(workflow) if not callable(parse_input): raise TypeError("parse_input must be a callable.") + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) super().__init__( - workflow=workflow, - parse_input=parse_input, openapi_spec=openapi_spec, - checkpoint_store_provider=checkpoint_store_provider, - **kwargs, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) + if sse_keep_alive_interval_seconds is not None: + self.config.sse_keepalive_interval = sse_keep_alive_interval_seconds + self._parse_input = parse_input + self._checkpoint_storage_provider = ( + CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider + ) + self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( + weakref.WeakValueDictionary() ) + self.invoke_handler(self._handle_invoke) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + async def _parse(self, request: Request) -> WorkflowTurn[Any]: + result = self._parse_input(request) + parsed = await result if inspect.isawaitable(result) else result + if not isinstance(parsed, WorkflowTurn): + raise TypeError("parse_input must return WorkflowTurn.") + return parsed + + async def _handle_invoke(self, request: Request) -> Response: + """Invoke the native workflow with a trusted request.""" + context = get_request_context() + try: + hosted_scope = invocation_request_scope(self.config, request, context) + partition_key = invocation_partition_key(self.config, context=context, scope=hosted_scope) + except RuntimeError as exc: + logger.error("Failed to resolve Invocations session: %s", exc) + return JSONResponse({"error": str(exc)}, status_code=500) + + try: + turn = await self._parse(request) + except (TypeError, ValueError) as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception: + logger.exception("Failed to prepare Invocations request") + return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) + + scope = hosted_scope or FoundryRequestScope.from_context(self.config, context) + if turn.stream: + + async def stream_response() -> AsyncGenerator[str]: + token: Token[FoundryAgentRequestContext] | None = set_request_context(context) + events = self._workflow_events(request, turn, scope, context, partition_key) + try: + async for frame in events: + wire_frame = _sse(frame["type"], frame) + reset_request_context(token) + token = None + yield wire_frame + token = set_request_context(context) + reset_request_context(token) + token = None + yield _sse("done", {"session_id": scope.session_id}) + except Exception as exc: + logger.exception("Invocations workflow stream failed") + message, status, code = _workflow_failure(exc) + if token is not None: + reset_request_context(token) + token = None + yield _sse("error", {"message": message, "code": code, "status": status}) + finally: + if token is None: + token = set_request_context(context) + try: + with CancelScope(shield=True): + await events.aclose() + finally: + reset_request_context(token) + + return _WorkflowStreamingResponse(stream_response(), keep_alive=self.config.sse_keepalive_interval > 0) + + try: + frames = [frame async for frame in self._workflow_events(request, turn, scope, context, partition_key)] + return JSONResponse({"output": frames}) + except Exception as exc: + logger.exception("Invocations workflow request failed") + message, status, _ = _workflow_failure(exc) + return JSONResponse({"error": message}, status_code=status) + + async def _workflow_events( + self, + request: Request, + turn: WorkflowTurn[Any], + scope: FoundryRequestScope, + context: FoundryAgentRequestContext, + partition_key: str | tuple[str, str], + ) -> AsyncGenerator[dict[str, Any]]: + frames: list[dict[str, Any]] = [] + snapshot_bytes = 0 + deferred: list[dict[str, Any]] = [] + defer = False + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock: + workflow = await self._workflow_resolver.resolve(request) + run = await HostedWorkflowRun.prepare( + workflow, + scope=scope, + response_id=context.call_id or uuid.uuid4().hex, + config=self.config, + platform_context=context, + checkpoint_store_provider=self._checkpoint_storage_provider, + lineage_id="invocations", + stored=True, + replay_completed=True, + fresh_factory=True, + reclaim_superseded=True, + ) + if run.has_completed_output: + for frame in _workflow_snapshot_frames(run.snapshot): + yield frame + return + try: + turn = run.validate_turn(turn) + except (TypeError, ValueError) as exc: + raise _WorkflowRequestError(str(exc)) from exc + if ( + run.stored + and turn.input is not None + and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) + ): + try: + self._checkpoint_storage_provider.validate_checkpoint_value(turn.input) + except WorkflowCheckpointException as exc: + raise _WorkflowRequestError( + "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " + 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' + ) from exc + if ( + run.stored + and turn.responses is not None + and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) + ): + try: + for reply in turn.responses.values(): + self._checkpoint_storage_provider.validate_checkpoint_value(reply) + except WorkflowCheckpointException as exc: + raise _WorkflowRequestError( + "Stored native workflow reply uses an application type that is not checkpoint-allowlisted. " + 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' + ) from exc + committed = False + resources = AsyncExitStack() + try: + await run.claim() + try: + entered: set[int] = set() + for agent in workflow_agents(workflow): + if id(agent) not in entered and isinstance(agent, AbstractAsyncContextManager): + entered.add(id(agent)) + await resources.enter_async_context(agent) + events = run.events(turn) + resources.push_async_callback(events.aclose) + async for event in events: + frame = _workflow_frame(event) + if frame is None: + continue + frame_size = _workflow_frame_size(frame) + if ( + len(frames) >= _MAX_WORKFLOW_SNAPSHOT_EVENTS + or snapshot_bytes + frame_size > _MAX_WORKFLOW_SNAPSHOT_BYTES + ): + raise _WorkflowOutputError( + "Workflow output exceeds the bounded Invocations snapshot limit." + ) + snapshot_bytes += frame_size + frames.append(frame) + defer = defer or event.type == "request_info" + if defer: + deferred.append(frame) + else: + yield frame + finally: + # Starlette cancellation must not interrupt core/resource cleanup. + with CancelScope(shield=True): + await resources.__aexit__(*sys.exc_info()) + await run.commit({"output": frames}) + committed = True + finally: + if not committed: + failure = sys.exc_info()[1] + try: + with CancelScope(shield=True): + await run.abort() + except Exception as exc: + logger.exception("Failed to persist interrupted Invocations workflow state") + if failure is None: + raise + if isinstance(failure, Exception): + raise RuntimeError( + "Workflow invocation failed and its interrupted state could not be persisted." + ) from exc + # Pending reply authority is durable, and a reply need not wait on this stream's lock. + for frame in deferred: + yield frame diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py index 1be4e3109b3..f408189295d 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py @@ -75,7 +75,6 @@ _reject_busy_conversation, # pyright: ignore[reportPrivateUsage] _set_agent_session, # pyright: ignore[reportPrivateUsage] _SignalledIterator, # pyright: ignore[reportPrivateUsage] - _validate_response_server_options, # pyright: ignore[reportPrivateUsage] consent_url_from_error, ) from ._scope import FoundryRequestScope @@ -84,6 +83,7 @@ OptionsHook, ResponsesRequest, UnsupportedOptions, + _build_response_server_options, # pyright: ignore[reportPrivateUsage] is_agent, prepare_response_options, resolve_agent, @@ -224,16 +224,20 @@ def __init__( agent: AgentSource[ResponsesRequest], *, prefix: str = "", - options: ResponsesServerOptions | None = None, response_store: ResponseProviderProtocol | None = None, agent_session_store_provider: StoreProvider[SessionStore] | None = None, function_approval_store_provider: StoreProvider[FunctionApprovalStore] | None = None, allowed_oauth_consent_origins: Sequence[str] | None = None, history_source: Literal["host_managed", "agent_managed", "inference_service"] = "host_managed", background_source: Literal["host_managed", "inference_service"] = "host_managed", + max_history_items: int | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + resilient_background: bool = False, prepare_options: OptionsHook | None = None, unsupported_options: UnsupportedOptions = "warn", - **kwargs: Any, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, ) -> None: """Initialize a ResponsesHostServer. @@ -246,8 +250,9 @@ def __init__( that keep mutable state outside `AgentSession`. `WorkflowAgent` instances and factory results are rejected. Use `ResponsesWorkflowHostServer` with a native workflow factory instead. + + Keyword Args: prefix: The URL prefix for the server. - options: Optional server options. response_store: Optional response store for caller-facing persistence and history. agent_session_store_provider: Optional provider for MAF agent session storage. If not provided, a default `AgentSessionStoreProvider` will be used. @@ -284,10 +289,25 @@ def __init__( client into provider background polling and requires `history_source="inference_service"`. Each poll retains the caller's run options and `background=True`; local tools must be idempotent because a crash before the next private token is saved can replay them. + max_history_items: Positive cap on prior outer history items, only with host-managed history. + `None` leaves the installed SDK default unchanged; it does not mean unlimited. + The cap does not limit tokens, turns, retention, or workflow checkpoints. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + shutdown_grace_period_seconds: Positive shutdown wait in seconds, defaulting to 10. + The SDK also uses it when DELETE waits for response persistence. Not a request timeout. + resilient_background: Permit SDK recovery of eligible outer background responses, + defaulting to `False`. The SDK's `resilient_background` setting is provider-independent. + This regular-agent adapter currently recovers only saved inference-service continuations, + so it requires inference-service history and background. This does not start background + execution or guarantee exactly-once effects. prepare_options: Developer hook to remove or replace caller model options for an agent. unsupported_options: `"warn"` (default), `"error"`, or `"ignore"` when a custom agent cannot accept runtime model options. - **kwargs: Additional keyword arguments. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. Note: The *caller* controls outer persistence with `POST /responses` `store=True/False`. @@ -311,9 +331,7 @@ def __init__( For provider background, only a saved private continuation token can be polled after a crash; a crash before the token is saved cannot be replayed safely. Other regular-agent runs are not crash-recoverable. - 5. Steering is temporarily unavailable for all agents. `steerable_conversations=True` fails - at construction, before starting a host or enabling the process-wide TaskManager. The current - AgentServer SDK retains futures for rejected turns after the steering queue fills. + 5. Steering is unavailable. Framework-generated SDK options keep steering disabled. 6. A stored, named `"inference_service"` or `"agent_managed"` conversation is claimed in the scoped session store before the agent runs. Conditional writes prevent a concurrent turn from mutating the same provider thread. If the turn fails or is cancelled, start a new conversation instead of @@ -321,25 +339,33 @@ def __init__( Raises: TypeError: If the agent is a `WorkflowAgent`, or native workflow settings are supplied. - ValueError: If the history, background, or unsupported-options policy is invalid. + ValueError: If a host setting or its history/background applicability is invalid. RuntimeError: If the agent configuration conflicts with the selected history source, - `resilient_background=True` is requested for a regular agent without provider background, or - `steerable_conversations=True` is requested while steering is unavailable. + or the agent does not support the selected inference-service background mode. """ - if "store" in kwargs: - raise TypeError("The constructor store= alias was removed; use response_store= instead.") - _validate_response_server_options(options) if history_source not in ("host_managed", "agent_managed", "inference_service"): raise ValueError("history_source must be 'host_managed', 'agent_managed', or 'inference_service'.") if background_source not in ("host_managed", "inference_service"): raise ValueError("background_source must be 'host_managed' or 'inference_service'.") if background_source == "inference_service" and history_source != "inference_service": raise ValueError("Provider background requires history_source='inference_service'.") - if background_source == "inference_service" and options and options.steerable_conversations: - raise ValueError("Provider background and steerable_conversations cannot be combined.") - if {"workflow", "parse_input", "parse_response", "checkpoint_store_provider"}.intersection(kwargs): - raise TypeError("Use ResponsesWorkflowHostServer for native workflows and checkpoint storage.") + if max_history_items is not None and history_source != "host_managed": + raise ValueError("max_history_items requires history_source='host_managed'.") + if resilient_background and background_source != "inference_service": + raise ValueError( + "resilient_background=True requires history_source='inference_service' and " + "background_source='inference_service' for regular agents." + ) validate_agent_source(agent) + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + unsupported_options = validate_unsupported_options(unsupported_options) + options = _build_response_server_options( + max_history_items=max_history_items, + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=resilient_background, + ) resolved_agent = agent if is_agent(agent) else None configuration = ( @@ -354,12 +380,9 @@ def __init__( prefix=prefix, options=options, store=response_store, - **kwargs, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), ) - if options and options.steerable_conversations: - from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled - - set_resilient_tasks_enabled(True) self._agent_source = agent self._agent = resolved_agent @@ -367,12 +390,12 @@ def __init__( self._history_source: _HistorySource = history_source self._background_source: Literal["host_managed", "inference_service"] = background_source self._prepare_options = prepare_options - self._unsupported_options = validate_unsupported_options(unsupported_options) + self._unsupported_options = unsupported_options self._host_options = options self._uses_agent_server_history = ( configuration.agent_server_history if configuration is not None else history_source == "host_managed" ) - self._resilient_background = bool(options and options.resilient_background) + self._resilient_background = resilient_background if resolved_agent is not None and configuration is not None: _initialize_agent_history(resolved_agent, configuration) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py index 1298d66ad85..8467c1ac641 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py @@ -47,7 +47,6 @@ from agent_framework.observability import OBSERVABILITY_SETTINGS from azure.ai.agentserver.responses import ( ResponseContext, - ResponsesServerOptions, ) from azure.ai.agentserver.responses._id_generator import IdGenerator from azure.ai.agentserver.responses.aio import ResponseEventStream @@ -789,18 +788,6 @@ def consent_url_from_error(exc: BaseException) -> list[ConsentError] | None: # endregion Foundry Toolbox Auth integration -def _validate_response_server_options( # pyright: ignore[reportUnusedFunction] - options: ResponsesServerOptions | None, -) -> None: - if options and options.steerable_conversations: - # TODO(foundry-hosting): Remove after Azure/azure-sdk-for-python#49233 ships and queue overflow is verified. - raise RuntimeError( - "steerable_conversations=True is temporarily unavailable: the current AgentServer SDK " - "retains futures for rejected steering turns. Wait for the official fix in " - "Azure/azure-sdk-for-python#49233, then update the dependency and verify queue overflow." - ) - - # region Active Builder State diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py index 31f148331df..8b1805073ff 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py @@ -14,7 +14,7 @@ from contextvars import ContextVar from dataclasses import dataclass, fields, is_dataclass from datetime import date, datetime -from typing import Any, cast +from typing import Any, Literal, cast from agent_framework import ( AgentResponse, @@ -32,7 +32,7 @@ from agent_framework._telemetry import mark_feature_used from anyio import CancelScope from azure.ai.agentserver.core import get_request_context -from azure.ai.agentserver.responses import ResponseContext, ResponseProviderProtocol, ResponsesServerOptions +from azure.ai.agentserver.responses import ResponseContext, ResponseProviderProtocol from azure.ai.agentserver.responses._id_generator import IdGenerator from azure.ai.agentserver.responses.aio import ResponseEventStream from azure.ai.agentserver.responses.hosting import ResponsesAgentServerHost @@ -54,7 +54,6 @@ _OutputItemTracker, # pyright: ignore[reportPrivateUsage] _request_without_reserved_metadata, # pyright: ignore[reportPrivateUsage] _SignalledIterator, # pyright: ignore[reportPrivateUsage] - _validate_response_server_options, # pyright: ignore[reportPrivateUsage] ) from ._scope import FoundryRequestScope from ._shared import ( @@ -63,6 +62,7 @@ WorkflowResolver, WorkflowSource, WorkflowTurn, + _build_response_server_options, # pyright: ignore[reportPrivateUsage] prepare_response_options, prepare_workflow_kwargs, response_run_options, @@ -337,12 +337,15 @@ def __init__( *, parse_input: Callable[[ResponsesRequest], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], prefix: str = "", - options: ResponsesServerOptions | None = None, response_store: ResponseProviderProtocol | None = None, checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, prepare_options: OptionsHook | None = None, allowed_oauth_consent_origins: Sequence[str] | None = None, - **kwargs: Any, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + resilient_background: bool = False, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, ) -> None: """Create a native workflow host with explicit parsing and checkpoint ownership. @@ -356,8 +359,6 @@ def __init__( parse_input: Sync or async parser of this turn's `ResponsesRequest`. Return a typed `WorkflowTurn(input=...)` or a complete batch of validated pending `responses`. prefix: URL prefix for Responses routes. - options: AgentServer options, including outer background recovery. Steering remains disabled. - Outer background execution does not enable provider background on workflow agents. response_store: Backend for caller-facing Responses persistence, not the wire-level `store` flag. checkpoint_store_provider: Scoped checkpoint storage. Defaults to `CheckpointStoreProvider`. Native workflow history and pending replies are checkpoint-owned, not agent-session-owned. @@ -366,47 +367,52 @@ def __init__( This does not enable inner provider background or service-owned history. allowed_oauth_consent_origins: Optional exact HTTPS origins for consent links. Omission retains safe-HTTPS validation; an empty sequence rejects all origins. - **kwargs: Additional AgentServer host settings forwarded to the SDK. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + shutdown_grace_period_seconds: Positive shutdown wait in seconds, defaulting to 10. + The SDK also uses it when DELETE waits for response persistence. Not a request timeout. + resilient_background: Permit checkpoint-backed outer workflow recovery, defaulting to + `False`. This does not enable inner provider background or guarantee exactly-once effects. + Framework-generated SDK options keep steering disabled. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. Raises: TypeError: If the workflow is not a request-aware factory, the parser is not callable, or an agent-only setting is supplied. - RuntimeError: If steering is requested while unavailable. + ValueError: If a supported host setting has an invalid value. """ - if "parse_response" in kwargs: - raise TypeError("parse_response was renamed to parse_input.") - forbidden = { - "agent", - "agent_session_store_provider", - "function_approval_store_provider", - "history_source", - "background_source", - "unsupported_options", - "store", - }.intersection(kwargs) - if forbidden: - raise TypeError(f"Unsupported native workflow host settings: {', '.join(sorted(forbidden))}.") validate_workflow_source(workflow) if not callable(parse_input): raise TypeError("parse_input must be a callable.") if prepare_options is not None and not callable(prepare_options): raise TypeError("prepare_options must be a callable.") - _validate_response_server_options(options) allowed_origins = _normalize_allowed_oauth_consent_origins(allowed_oauth_consent_origins) - super().__init__(prefix=prefix, options=options, store=response_store, **kwargs) + options = _build_response_server_options( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=resilient_background, + ) + super().__init__( + prefix=prefix, + options=options, + store=response_store, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) self._resolver = WorkflowResolver(workflow) self._parse_input = parse_input self._checkpoint_store_provider = ( CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider ) self._prepare_options = prepare_options - self._resilient_background = bool(options and options.resilient_background) + self._resilient_background = resilient_background self._allowed_origins = allowed_origins self._bind_streaming_route( prefix=prefix, - keep_alive=bool( - (options and options.sse_keep_alive_interval_seconds) or self.config.sse_keepalive_interval - ), + keep_alive=(sse_keep_alive_interval_seconds or self.config.sse_keepalive_interval) > 0, ) self.response_handler(self._handle_response) mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py index 504bc3b0a8b..09b316b995e 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py @@ -33,7 +33,7 @@ WorkflowInvocationKwargs, WorkflowRunState, ) -from azure.ai.agentserver.responses import ResponseContext +from azure.ai.agentserver.responses import ResponseContext, ResponsesServerOptions from azure.ai.agentserver.responses.models import CreateResponse, Item from typing_extensions import TypeVar @@ -42,6 +42,46 @@ UnsupportedOptions: TypeAlias = Literal["ignore", "warn", "error"] +def validate_sse_keep_alive_interval(value: int | None) -> None: + """Validate an explicit heartbeat interval without replacing SDK environment resolution.""" + if value is not None: + if isinstance(value, bool) or not isinstance(value, int): + raise TypeError("sse_keep_alive_interval_seconds must be an integer or None.") + if value <= 0: + raise ValueError("sse_keep_alive_interval_seconds must be positive when supplied.") + + +def _build_response_server_options( # pyright: ignore[reportUnusedFunction] + *, + max_history_items: int | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + resilient_background: bool = False, +) -> ResponsesServerOptions: + """Build supported SDK settings, leaving an omitted history limit at its SDK default.""" + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + if isinstance(shutdown_grace_period_seconds, bool) or not isinstance(shutdown_grace_period_seconds, int): + raise TypeError("shutdown_grace_period_seconds must be an integer.") + if shutdown_grace_period_seconds <= 0: + raise ValueError("shutdown_grace_period_seconds must be positive.") + if not isinstance(resilient_background, bool): + raise TypeError("resilient_background must be a boolean.") + history_options: dict[str, Any] = {} + if max_history_items is not None: + if isinstance(max_history_items, bool) or not isinstance(max_history_items, int): + raise TypeError("max_history_items must be an integer or None.") + if max_history_items <= 0: + raise ValueError("max_history_items must be positive when supplied.") + history_options["default_fetch_history_count"] = max_history_items + return ResponsesServerOptions( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=resilient_background, + steerable_conversations=False, + **history_options, + ) + + InputT = TypeVar("InputT", default=Any) diff --git a/python/packages/foundry_hosting/tests/conftest.py b/python/packages/foundry_hosting/tests/conftest.py index abf0234046a..3348c068c6f 100644 --- a/python/packages/foundry_hosting/tests/conftest.py +++ b/python/packages/foundry_hosting/tests/conftest.py @@ -2,12 +2,24 @@ """Fixtures for Foundry Hosting tests.""" +from __future__ import annotations + from collections.abc import Iterator +from functools import partial from pathlib import Path from tempfile import TemporaryDirectory +from typing import TYPE_CHECKING, Any import pytest +if TYPE_CHECKING: + from agent_framework_foundry_hosting import ( + InvocationsHostServer, + InvocationsWorkflowHostServer, + ResponsesHostServer, + ResponsesWorkflowHostServer, + ) + @pytest.fixture(autouse=True) def disable_agentserver_external_telemetry(monkeypatch: pytest.MonkeyPatch) -> None: @@ -22,3 +34,36 @@ def isolate_local_agentserver_state_root(monkeypatch: pytest.MonkeyPatch) -> Ite with TemporaryDirectory(prefix="agentserver-test-") as directory: monkeypatch.setenv("AGENTSERVER_STATE_ROOT", str(Path(directory) / "agentserver-state")) yield + + +@pytest.fixture( + params=[ + "ResponsesHostServer", + "ResponsesWorkflowHostServer", + "InvocationsHostServer", + "InvocationsWorkflowHostServer", + ] +) +def host_type( + request: pytest.FixtureRequest, +) -> type[ResponsesHostServer | ResponsesWorkflowHostServer | InvocationsHostServer | InvocationsWorkflowHostServer]: + import agent_framework_foundry_hosting + + return getattr(agent_framework_foundry_hosting, request.param) + + +@pytest.fixture +def host_constructor( + host_type: type[ + ResponsesHostServer | ResponsesWorkflowHostServer | InvocationsHostServer | InvocationsWorkflowHostServer + ], +) -> partial[Any]: + settings: dict[str, Any] + if "Workflow" in host_type.__name__: + settings = { + "workflow": lambda request: pytest.fail("Construction must not invoke the workflow factory."), + "parse_input": lambda request: pytest.fail("Construction must not invoke the parser."), + } + else: + settings = {"agent": lambda request: pytest.fail("Construction must not invoke the agent factory.")} + return partial(host_type, **settings) diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py new file mode 100644 index 00000000000..8924d81a33c --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py @@ -0,0 +1,182 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Verify explicit framework settings and the unvalidated AgentServer constructor boundary.""" + +from __future__ import annotations + +from collections.abc import Mapping +from functools import partial +from types import MappingProxyType +from typing import Any +from unittest.mock import patch + +import pytest +from azure.ai.agentserver.responses import ResponsesServerOptions + + +@pytest.mark.parametrize( + "settings", + [ + None, + {}, + MappingProxyType({}), + {"default_model": "sdk-owned", "custom_sdk_setting": {"value": object()}}, + MappingProxyType({"steerable_conversations": True, "custom_sdk_setting": object()}), + ], +) +def test_agent_server_kwargs_are_forwarded_without_validation_or_mutation( + host_type: type[Any], host_constructor: partial[Any], settings: Mapping[str, Any] | None +) -> None: + before = dict(settings or {}) + sdk_host = host_type.__bases__[0] + with ( + patch.object(sdk_host, "__init__", side_effect=RuntimeError("SDK construction failed")) as initialize, + pytest.raises(RuntimeError, match="SDK construction failed"), + ): + host_constructor(agent_server_kwargs=settings) + initialize.assert_called_once() + forwarded = initialize.call_args.kwargs + assert "agent_server_kwargs" not in forwarded + for key, value in before.items(): + assert forwarded[key] is value + assert dict(settings or {}) == before + + +@pytest.mark.parametrize("log_level", [None, "DEBUG"]) +def test_log_level_is_forwarded_unchanged_to_agent_server( + host_type: type[Any], host_constructor: partial[Any], log_level: str | None +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__", side_effect=RuntimeError("SDK boundary")) as initialize, + pytest.raises(RuntimeError, match="SDK boundary"), + ): + host_constructor(log_level=log_level) + assert initialize.call_args.kwargs["log_level"] == log_level + + +@pytest.mark.parametrize("interval", [True, 1.5, "5", 0, -1]) +def test_invalid_explicit_heartbeats_fail_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any], interval: Any +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises((TypeError, ValueError), match="sse_keep_alive_interval_seconds"), + ): + host_constructor(sse_keep_alive_interval_seconds=interval) + initialize.assert_not_called() + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer", "ResponsesWorkflowHostServer"], indirect=True) +@pytest.mark.parametrize( + "settings", + [ + {"shutdown_grace_period_seconds": True}, + {"shutdown_grace_period_seconds": 0}, + {"shutdown_grace_period_seconds": -1}, + {"resilient_background": None}, + {"resilient_background": 1}, + ], +) +def test_invalid_responses_settings_fail_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any], settings: dict[str, Any] +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises((TypeError, ValueError), match=next(iter(settings))), + ): + host_constructor(**settings) + initialize.assert_not_called() + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer"], indirect=True) +@pytest.mark.parametrize("limit", [None, 3]) +def test_history_limit_inherits_sdk_default_or_sets_an_explicit_cap( + host_type: type[Any], host_constructor: partial[Any], monkeypatch: pytest.MonkeyPatch, limit: int | None +) -> None: + monkeypatch.setenv("DEFAULT_FETCH_HISTORY_ITEM_COUNT", "7") + with ( + patch.object(host_type.__bases__[0], "__init__", side_effect=RuntimeError("SDK boundary")) as initialize, + pytest.raises(RuntimeError, match="SDK boundary"), + ): + host_constructor(max_history_items=limit) + options = initialize.call_args.kwargs["options"] + expected = ResponsesServerOptions().default_fetch_history_count if limit is None else limit + assert options.default_fetch_history_count == expected + assert options.default_model is None + assert options.steerable_conversations is False + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer", "ResponsesWorkflowHostServer"], indirect=True) +def test_response_operational_settings_reach_the_sdk(host_type: type[Any], host_constructor: partial[Any]) -> None: + settings: dict[str, Any] = { + "sse_keep_alive_interval_seconds": 3, + "shutdown_grace_period_seconds": 2, + "resilient_background": True, + } + if host_type.__name__ == "ResponsesHostServer": + settings.update(history_source="inference_service", background_source="inference_service") + with ( + patch.object(host_type.__bases__[0], "__init__", side_effect=RuntimeError("SDK boundary")) as initialize, + pytest.raises(RuntimeError, match="SDK boundary"), + ): + host_constructor(**settings) + options = initialize.call_args.kwargs["options"] + assert options.sse_keep_alive_interval_seconds == 3 + assert options.shutdown_grace_period_seconds == 2 + assert options.resilient_background is True + + +@pytest.mark.parametrize("host_type", ["InvocationsHostServer", "InvocationsWorkflowHostServer"], indirect=True) +@pytest.mark.parametrize("environment, explicit, expected", [(None, None, 0), ("3", None, 3), ("3", 5, 5)]) +def test_invocations_heartbeat_configuration_is_per_host( + host_constructor: partial[Any], + monkeypatch: pytest.MonkeyPatch, + environment: str | None, + explicit: int | None, + expected: int, +) -> None: + if environment is None: + monkeypatch.delenv("SSE_KEEPALIVE_INTERVAL", raising=False) + else: + monkeypatch.setenv("SSE_KEEPALIVE_INTERVAL", environment) + configured = host_constructor(sse_keep_alive_interval_seconds=explicit) + inherited = host_constructor() + assert configured.config.sse_keepalive_interval == expected + assert inherited.config.sse_keepalive_interval == int(environment or "0") + assert configured.config is not inherited.config + + +@pytest.mark.parametrize("host_type", ["InvocationsHostServer"], indirect=True) +def test_legacy_agent_stream_cannot_enable_heartbeats( + host_constructor: partial[Any], monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setenv("SSE_KEEPALIVE_INTERVAL", "3") + with pytest.warns(DeprecationWarning): + server = host_constructor(legacy_wire_format=True) + assert server.config.sse_keepalive_interval == 0 + with pytest.raises(ValueError, match="framed SSE"): + host_constructor(legacy_wire_format=True, sse_keep_alive_interval_seconds=1) + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer"], indirect=True) +@pytest.mark.parametrize( + "settings", + [ + {"max_history_items": True}, + {"max_history_items": 1.5}, + {"max_history_items": 0}, + {"max_history_items": -1}, + {"max_history_items": 3, "history_source": "agent_managed"}, + {"max_history_items": 3, "history_source": "inference_service"}, + {"resilient_background": True}, + ], +) +def test_agent_setting_applicability_fails_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any], settings: dict[str, Any] +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises((TypeError, ValueError), match=next(iter(settings))), + ): + host_constructor(**settings) + initialize.assert_not_called() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py index 81ef021147a..ca2f762f647 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py @@ -313,7 +313,7 @@ def _host( for value in (Ticket, TicketModel, TicketState, TicketReview, TicketDecision) ] ), - configure_observability=None, + agent_server_kwargs={"configure_observability": None}, ) @@ -439,14 +439,14 @@ def create(_request: Request) -> Workflow: return WorkflowBuilder(name="tickets", start_executor=_TicketExecutor([])).build() invalid_agent: Any = object() - with pytest.raises(TypeError, match="InvocationsWorkflowHostServer"): - InvocationsHostServer(agent=invalid_agent, workflow=create) + with pytest.raises(TypeError, match="workflow"): + cast(Any, InvocationsHostServer)(agent=invalid_agent, workflow=create) with pytest.raises(TypeError, match="parse_input"): cast(Any, InvocationsWorkflowHostServer)(workflow=create) def test_workflow_rejects_agent_legacy_wire_format(self) -> None: with pytest.raises(TypeError, match="legacy_wire_format"): - InvocationsWorkflowHostServer( + cast(Any, InvocationsWorkflowHostServer)( workflow=lambda _: WorkflowBuilder(name="tickets", start_executor=_TicketExecutor([])).build(), parse_input=_parse_ticket, legacy_wire_format=True, @@ -1273,11 +1273,11 @@ async def handle(self, _: Ticket, ctx: WorkflowContext[None, str]) -> None: byte_limit = 4096 if limit_kind == "events" else 128 with ( patch( - "agent_framework_foundry_hosting._invocations_shared._MAX_WORKFLOW_SNAPSHOT_EVENTS", + "agent_framework_foundry_hosting._invocations_workflow._MAX_WORKFLOW_SNAPSHOT_EVENTS", event_limit, ), patch( - "agent_framework_foundry_hosting._invocations_shared._MAX_WORKFLOW_SNAPSHOT_BYTES", + "agent_framework_foundry_hosting._invocations_workflow._MAX_WORKFLOW_SNAPSHOT_BYTES", byte_limit, ), ): diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py index 14d9b1f11d3..4f9f218aa45 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py @@ -35,7 +35,7 @@ ) from agent_framework._workflows._agent_utils import prepare_executor_run_kwargs from agent_framework._workflows._const import RESOLVED_WORKFLOW_RUN_KWARGS_KEY, WORKFLOW_RUN_KWARGS_KEY -from azure.ai.agentserver.responses import FileResponseStore, InMemoryResponseProvider, ResponsesServerOptions +from azure.ai.agentserver.responses import FileResponseStore, InMemoryResponseProvider from azure.ai.agentserver.responses._id_generator import IdGenerator from pydantic import BaseModel @@ -253,7 +253,7 @@ def _server( checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{_Ticket.__module__}:{_Ticket.__qualname__}"], ), - configure_observability=None, + agent_server_kwargs={"configure_observability": None}, **kwargs, ) @@ -349,7 +349,7 @@ async def test_default_checkpoint_provider_rejects_unallowlisted_typed_input_bef workflow=lambda request: _build(), parse_input=_parse, response_store=InMemoryResponseProvider(), - configure_observability=None, + agent_server_kwargs={"configure_observability": None}, ) response = await _post(server, '{"text":"typed"}') @@ -486,7 +486,7 @@ async def test_unsupported_shared_workflow_is_not_cloned_or_unwrapped() -> None: ResponsesWorkflowHostServer( workflow=cast(Any, _build()), parse_input=_parse, - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ) @@ -1052,8 +1052,13 @@ async def update_response(self, response: Any, **kwargs: Any) -> None: provider = FailingCheckpointStore(storage_dir=tmp_path / "responses") factory = _progress_factory(started, release, stopped) - options = ResponsesServerOptions(resilient_background=True, shutdown_grace_period_seconds=1) - server = _server(factory, parser=_parse_count, response_store=provider, options=options) + server = _server( + factory, + parser=_parse_count, + response_store=provider, + resilient_background=True, + shutdown_grace_period_seconds=1, + ) paired = asyncio.Event() original = HostedWorkflowRun.stage @@ -1090,7 +1095,8 @@ async def stage(run: HostedWorkflowRun, snapshot: Any, **kwargs: Any) -> None: factory, parser=_parse_count, response_store=provider, - options=ResponsesServerOptions(resilient_background=True, shutdown_grace_period_seconds=1), + resilient_background=True, + shutdown_grace_period_seconds=1, ) async with ( restarted.router.lifespan_context(restarted), diff --git a/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py b/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py index e3aa46d553f..44f6ac80b9e 100644 --- a/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py +++ b/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py @@ -55,8 +55,8 @@ def test_protocol_hosts_import_independently_in_either_order(protocol: str, work from agent_framework_foundry_hosting import {names[1]} from azure.ai.agentserver.{sdk_module} import {base} - assert issubclass({agent}, {base}) - assert issubclass({workflow}, {base}) + assert {agent}.__bases__ == ({base},) + assert {workflow}.__bases__ == ({base},) """ ) subprocess.run([sys.executable, "-I", "-c", code], check=True) @@ -102,6 +102,11 @@ def test_host_signatures_expose_only_the_selected_execution_model(protocol: str) for name in ("agent", "agent_session_store_provider", "history_source", "background_source", "unsupported_options"): assert name not in workflow_parameters assert ("prepare_options" in workflow_parameters) is (protocol == "Responses") + for parameters in (agent_parameters, workflow_parameters): + for name in ("log_level", "agent_server_kwargs"): + assert parameters[name].default is None + assert parameters[name].kind is inspect.Parameter.KEYWORD_ONLY + assert all(parameter.kind is not inspect.Parameter.VAR_KEYWORD for parameter in parameters.values()) if protocol == "Responses": assert "response_store" in agent_parameters assert "store" not in agent_parameters @@ -203,17 +208,15 @@ def test_legacy_parser_names_are_not_silently_forwarded_to_sdk(name: str, legacy else "azure.ai.agentserver.invocations.InvocationAgentServerHost" ) with patch(f"{base}.__init__", return_value=None) as initialize: - with pytest.raises(TypeError, match=f"{legacy} was renamed to parse_input"): + with pytest.raises(TypeError, match=legacy): host(**values) initialize.assert_not_called() -def test_responses_workflow_host_preserves_steering_guard() -> None: - from azure.ai.agentserver.responses import ResponsesServerOptions - - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable"): +def test_responses_workflow_host_rejects_removed_options() -> None: + with pytest.raises(TypeError, match="options"): cast(Any, hosting.ResponsesWorkflowHostServer)( workflow=lambda _: None, parse_input=lambda _: None, - options=ResponsesServerOptions(steerable_conversations=True), + options=None, ) diff --git a/python/packages/foundry_hosting/tests/test_invocations.py b/python/packages/foundry_hosting/tests/test_invocations.py index 3a488f95365..25781a54c83 100644 --- a/python/packages/foundry_hosting/tests/test_invocations.py +++ b/python/packages/foundry_hosting/tests/test_invocations.py @@ -3,7 +3,7 @@ """Unit tests for InvocationsHostServer. These tests exercise ``InvocationsHostServer`` directly by constructing the -host, driving ``_partition_key`` and ``_handle_invoke`` with a fake agent and +host, driving the shared partition-key helper and ``_handle_invoke`` with a fake agent and mock requests. The Foundry request context is injected via the public ``set_request_context`` / ``reset_request_context`` helpers rather than by patching, matching the style used in ``test_toolbox.py``. @@ -60,6 +60,7 @@ from starlette.responses import Response, StreamingResponse from agent_framework_foundry_hosting import InvocationRun, InvocationsHostServer, StoreProvider +from agent_framework_foundry_hosting._invocations_shared import invocation_partition_key from agent_framework_foundry_hosting._state_store import FoundryAgentSessionStore pytestmark = pytest.mark.filterwarnings("ignore:.*SessionStore is experimental.*") @@ -1425,17 +1426,17 @@ class TestPartitionKey: def test_local_returns_session_id(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) with _request_context(session_id="sess-1"): - assert server._partition_key() == "sess-1" # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == "sess-1" def test_local_missing_session_id_raises(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) with _request_context(), pytest.raises(RuntimeError, match="missing session_id"): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) def test_local_ignores_user_id(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) with _request_context(session_id="sess-1", user_id="user-1"): - assert server._partition_key() == "sess-1" # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == "sess-1" @pytest.mark.parametrize("user_id", [None, ""]) def test_hosted_requires_user_id(self, user_id: str | None) -> None: @@ -1446,21 +1447,21 @@ def test_hosted_requires_user_id(self, user_id: str | None) -> None: _request_context(call_id="call-1", session_id="sess-1", user_id=user_id), pytest.raises(RuntimeError, match="missing user_id"), ): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) def test_hosted_returns_composite_key(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) server.config.is_hosted = True server.config.session_id = "sess-1" with _request_context(call_id="call-1", session_id="sess-1", user_id="user-1"): - assert server._partition_key() == ("sess-1", "user-1") # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == ("sess-1", "user-1") def test_hosted_uses_platform_session_when_context_omits_it(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) server.config.is_hosted = True server.config.session_id = "sess-1" with _request_context(call_id="call-1", user_id="user-1"): - assert server._partition_key() == ("sess-1", "user-1") # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == ("sess-1", "user-1") def test_hosted_rejects_a_different_request_session(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) @@ -1470,7 +1471,7 @@ def test_hosted_rejects_a_different_request_session(self) -> None: _request_context(call_id="call-1", session_id="caller-session", user_id="user-1"), pytest.raises(RuntimeError, match="does not match"), ): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) def test_hosted_requires_platform_call_id(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) @@ -1480,7 +1481,7 @@ def test_hosted_requires_platform_call_id(self) -> None: _request_context(session_id="sess-1", user_id="user-1"), pytest.raises(RuntimeError, match="trusted user ID and call ID"), ): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) async def test_hosted_keys_and_session_ids_preserve_identifier_values(self) -> None: agent = _make_agent(response_text="hi") @@ -1493,7 +1494,7 @@ async def test_hosted_keys_and_session_ids_preserve_identifier_values(self) -> N for session_id, user_id in product(identifiers, repeat=2): server.config.session_id = session_id with _request_context(call_id="call-1", session_id=session_id, user_id=user_id): - key = server._partition_key() # pyright: ignore[reportPrivateUsage] + key = invocation_partition_key(server.config) response = await server._handle_invoke(request) # pyright: ignore[reportPrivateUsage] assert isinstance(key, tuple) @@ -1946,7 +1947,7 @@ async def test_reusing_session_restores_identifier(self, hosted: bool, stream: b with ( _request_context(call_id="call-1", session_id="sess-1", user_id="user-1"), patch( - "agent_framework_foundry_hosting._invocations_shared.AgentSession", wraps=AgentSession + "agent_framework_foundry_hosting._invocations_agent.AgentSession", wraps=AgentSession ) as session_factory, ): for _ in range(2): diff --git a/python/packages/foundry_hosting/tests/test_responses.py b/python/packages/foundry_hosting/tests/test_responses.py index 04ae6bc1a38..7875fc02f59 100644 --- a/python/packages/foundry_hosting/tests/test_responses.py +++ b/python/packages/foundry_hosting/tests/test_responses.py @@ -1708,42 +1708,42 @@ def test_failed_init_does_not_mutate_agent(self) -> None: default_options={"store": True}, ) - with pytest.raises(RuntimeError, match="resilient_background"): + with pytest.raises(ValueError, match="resilient_background"): ResponsesHostServer( agent, - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ) assert agent.default_options["store"] is True assert agent.context_providers == [] - def test_init_rejects_resilient_background_for_non_workflow_agent(self, tmp_path: Path) -> None: + def test_init_rejects_background_recovery_without_provider_background(self, tmp_path: Path) -> None: agent = _make_agent( response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("hi")])]) ) - with pytest.raises(RuntimeError, match="resilient_background"): + with pytest.raises(ValueError, match="resilient_background"): ResponsesHostServer( agent, response_store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ) - def test_init_rejects_steerable_conversations(self) -> None: - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable"): - ResponsesHostServer( + def test_init_rejects_removed_options(self) -> None: + with pytest.raises(TypeError, match="options"): + cast(Any, ResponsesHostServer)( _make_agent(), response_store=InMemoryResponseProvider(), - options=ResponsesServerOptions(steerable_conversations=True), + options=None, ) def test_steering_rejected_before_enabling_task_manager_or_starting_host(self) -> None: agent_factory = MagicMock() def construct(_turn: int) -> str: - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable") as error: - ResponsesHostServer( + with pytest.raises(TypeError, match="steerable_conversations") as error: + cast(Any, ResponsesHostServer)( agent=agent_factory, - options=ResponsesServerOptions(steerable_conversations=True), + steerable_conversations=True, ) return str(error.value) @@ -1755,7 +1755,7 @@ def construct(_turn: int) -> str: failures = list(executor.map(construct, range(24))) assert len(failures) == 24 - assert all("Azure/azure-sdk-for-python#49233" in failure for failure in failures) + assert all("steerable_conversations" in failure for failure in failures) enable.assert_not_called() base_init.assert_not_called() agent_factory.assert_not_called() @@ -1764,7 +1764,7 @@ async def test_non_steerable_background_still_uses_outer_response_id(self) -> No agent = _make_agent( stream_updates=[AgentResponseUpdate(contents=[Content.from_text("finished")], role="assistant")] ) - server = _make_server(agent, options=ResponsesServerOptions(steerable_conversations=False)) + server = _make_server(agent) async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as http: pending = await http.post("/responses", json={"input": "hello", "store": True, "background": True}) @@ -1785,12 +1785,12 @@ def test_provider_background_rejects_steering_and_wrong_history_source(self) -> agent = Agent(client=_ServiceStorageRecordingClient()) with pytest.raises(ValueError, match="history_source='inference_service'"): _make_server(agent, background_source="inference_service") - with pytest.raises(RuntimeError, match="temporarily unavailable"): + with pytest.raises(TypeError, match="steerable_conversations"): _make_server( agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(steerable_conversations=True), + steerable_conversations=True, ) @pytest.mark.parametrize("old_store", [None, InMemoryResponseProvider()]) @@ -1803,7 +1803,7 @@ def test_removed_store_alias_is_rejected_before_sdk_construction( if with_response_store: kwargs["response_store"] = InMemoryResponseProvider() with patch("azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost.__init__") as initialize: - with pytest.raises(TypeError, match="store= alias was removed; use response_store="): + with pytest.raises(TypeError, match="store"): ResponsesHostServer(agent, **kwargs) initialize.assert_not_called() assert agent.context_providers == [] @@ -1984,6 +1984,19 @@ async def test_responses_history_is_not_duplicated_by_default_local_history(self assert InMemoryHistoryProvider.DEFAULT_SOURCE_ID not in stored.state assert "_foundry_responses_history" not in stored.state + @pytest.mark.parametrize("limit", [1, 3]) + async def test_explicit_history_cap_limits_prior_items_without_dropping_current_input(self, limit: int) -> None: + client = _RecordingHistoryClient() + server = _make_server(Agent(client=client), max_history_items=limit) + + first = await _post(server, input_text="first") + second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) + third = await _post(server, input_text="third", previous_response_id=second.json()["id"]) + + assert third.json()["status"] == "completed", third.json() + prior = ["first", "recorded", "second", "recorded"] + assert [message.text for message in client.calls[-1]] == [*prior[-limit:], "third"] + async def test_agent_server_history_disables_service_storage(self) -> None: client = _ServiceStorageRecordingClient() agent = Agent[ChatOptions]( @@ -2495,7 +2508,7 @@ async def run(messages: Any = None, *, session: AgentSession, **kwargs: Any) -> session_store=SessionStore(), history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -2555,7 +2568,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True, temperature=0.35) @@ -2629,7 +2642,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) context = ResponseContext( @@ -2665,7 +2678,7 @@ async def test_provider_recovery_without_saved_token_does_not_restart_job( session_store=SessionStore(), history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -2708,7 +2721,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse( @@ -2740,7 +2753,7 @@ async def run( session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) recovered_context = ResponseContext(response_id="outer-recover", mode_flags=MagicMock()) @@ -2790,7 +2803,7 @@ def make_host() -> ResponsesHostServer: agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -2876,7 +2889,7 @@ def make_host() -> ResponsesHostServer: agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3066,7 +3079,7 @@ async def set(self, session_id: str, session: AgentSession) -> None: session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) context = ResponseContext(response_id="outer-tool-response", mode_flags=MagicMock()) @@ -3209,7 +3222,7 @@ async def sleep(_seconds: float) -> None: session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) response_id = f"outer-{stage}-{interruption}" @@ -3325,7 +3338,7 @@ def make_host() -> ResponsesHostServer: agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3434,7 +3447,7 @@ def make_host() -> ResponsesHostServer: session_store=store, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3513,7 +3526,7 @@ async def run( session_store=FailingTokenStore() if phase == "save" else SessionStore(), history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) diff --git a/python/packages/foundry_hosting/tests/test_responses_int.py b/python/packages/foundry_hosting/tests/test_responses_int.py index 8fc83283720..a84c571fa38 100644 --- a/python/packages/foundry_hosting/tests/test_responses_int.py +++ b/python/packages/foundry_hosting/tests/test_responses_int.py @@ -45,7 +45,7 @@ ) from agent_framework.foundry import FoundryChatClient, FoundryChatOptions from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled -from azure.ai.agentserver.responses import InMemoryResponseProvider, ResponsesServerOptions +from azure.ai.agentserver.responses import InMemoryResponseProvider from azure.identity import AzureCliCredential from openai import AsyncOpenAI, DefaultAsyncHttpxClient @@ -1069,7 +1069,7 @@ async def parse_input(request: ResponsesRequest) -> WorkflowTurn[list[Message]]: server = ResponsesWorkflowHostServer( workflow=lambda request: build_workflow(sleep_seconds), parse_input=parse_input, - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ) server.run(host="127.0.0.1", port=port) diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py index 67e46fe0b7c..525dd45a00a 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py @@ -7,7 +7,6 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest -from azure.ai.agentserver.responses import ResponsesServerOptions from azure.identity import DefaultAzureCredential from dotenv import load_dotenv @@ -31,7 +30,7 @@ def main() -> None: agent=create_agent, history_source="inference_service", background_source="inference_service", - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py index a95415d83ce..bf8ad6125cf 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py @@ -15,7 +15,6 @@ WorkflowTurn, ) from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled -from azure.ai.agentserver.responses import ResponsesServerOptions from pydantic import BaseModel, Field @@ -72,7 +71,7 @@ def main() -> None: checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{CountdownRequest.__module__}:{CountdownRequest.__qualname__}"], ), - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md index f0b2f5526fa..5d4214b67d5 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md @@ -1,12 +1,12 @@ # Long-running Responses agent (steering temporarily unavailable) -**Steering is currently gated.** `ResponsesHostServer` rejects -`ResponsesServerOptions(steerable_conversations=True)` during construction rather than starting a -TaskManager that can retain unbounded futures after rejected steering requests. The upstream fix is +**Steering is not exposed.** `ResponsesHostServer` has no `options` or `steerable_conversations` +parameter, and framework-generated SDK options keep steering disabled. The SDK TaskManager can retain +unbounded futures after rejected steering requests. The upstream fix is [Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233); -steering will be re-enabled only after a patched official `azure-ai-agentserver-core` wheel is released, -the package's minimum dependency and lockfile are updated, and a concurrent overflow test verifies -that rejected turns leave no pending futures. Do not enable steering using a private SDK patch or a +any future steering API needs a patched official `azure-ai-agentserver-core` wheel, updated minimum +dependencies and lockfile, and a concurrent overflow test proving that rejected turns leave no pending futures. +Do not enable steering using a private SDK patch or a queue-length precheck. [main.py](main.py) remains **runnable** as a regular non-steerable countdown agent. It uses diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py index 8169f5769b8..b6e3f20d040 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py @@ -4,8 +4,8 @@ The agent is asked to count down from a target number, pacing its own output with a short remark before each number so a response takes a while to fully generate. This sample currently runs -without steering; the hosting layer rejects ``steerable_conversations=True`` until a patched -AgentServer SDK is released and verified. +without steering; ``steerable_conversations`` is not a host parameter and framework-generated +SDK options keep steering disabled. Environment variables: FOUNDRY_PROJECT_ENDPOINT: Microsoft Foundry project endpoint. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py index f5c6b2f89f1..964d3b9ec2c 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py @@ -1,16 +1,17 @@ # Copyright (c) Microsoft. All rights reserved. -"""Verify that steering fails before starting a host while the AgentServer SDK is affected. +"""Verify that the unsupported steering setting cannot start an agent or host. -This script does not require Azure credentials or make model requests. The full end-to-end -steering verifier must be restored when a patched core wheel and overflow regression are in place. +This script does not require Azure credentials or make model requests. A future steering verifier +requires a supported steering API, a patched core wheel, and an overflow regression. Expected output: Steering is temporarily unavailable; no agent or host was started. """ +from typing import Any + from agent_framework import SupportsAgentRun from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest -from azure.ai.agentserver.responses import ResponsesServerOptions def _agent_factory(*, request: ResponsesRequest) -> SupportsAgentRun: @@ -19,10 +20,11 @@ def _agent_factory(*, request: ResponsesRequest) -> SupportsAgentRun: def main() -> None: """Confirm that the constructor rejects steering before creating an agent or host.""" + removed_settings: dict[str, Any] = {"steerable_conversations": True} try: - ResponsesHostServer(agent=_agent_factory, options=ResponsesServerOptions(steerable_conversations=True)) - except RuntimeError as exc: - if "steerable_conversations=True is temporarily unavailable" not in str(exc): + ResponsesHostServer(agent=_agent_factory, **removed_settings) + except TypeError as exc: + if "steerable_conversations" not in str(exc): raise print("Steering is temporarily unavailable; no agent or host was started.") return