diff --git a/docs/decisions/0055-python-foundry-hosting-contracts.md b/docs/decisions/0055-python-foundry-hosting-contracts.md new file mode 100644 index 00000000000..8a99f2a94a6 --- /dev/null +++ b/docs/decisions/0055-python-foundry-hosting-contracts.md @@ -0,0 +1,215 @@ +--- +status: proposed +date: 2026-11-10 +deciders: ["@eavanvalkenburg"] +--- + +# Python Foundry hosting contracts + +## Context and Problem Statement + +Foundry agent and native workflow hosting have different construction and state-ownership contracts. +A combined constructor exposes settings that apply to only one execution model and makes required +workflow inputs appear optional. Hosting also spans several distinct concerns: caller-visible +persistence, agent history, workflow checkpoints, downstream inference storage, background execution, +and recovery. + +The public API must make these boundaries explicit without duplicating protocol engines or implying +stronger persistence and execution guarantees than the underlying systems provide. + +## Decision Drivers + +- Make required inputs and state ownership visible in public signatures. +- Resolve identity-dependent resources from trusted request context. +- Isolate workflow execution state without cloning graphs or redesigning Core. +- Use consistent terminology while preserving protocol-specific request contracts. +- Expose supported configuration rather than the upstream SDK's entire options surface. + +## Considered Options + +- Separate agent and workflow hosts: explicit contracts, at the cost of more public entry points. +- One host with typed configuration groups: fewer entry points, but extra configuration layers. +- One flat constructor: fewer types, but conditional requirements and invalid setting combinations. + +## Decision Outcome + +Choose separate agent and native workflow hosts. Share protocol internals, but keep construction, +configuration, and state ownership specific to each execution model. + +The decisions below define the intended hosting contracts, not the implementation status of each feature. + +### Host boundaries + +Use `ResponsesHostServer` and `InvocationsHostServer` for agents, and +`ResponsesWorkflowHostServer` and `InvocationsWorkflowHostServer` for native workflows. +Agent hosts reject `WorkflowAgent`; workflow hosts require a factory returning a native `Workflow`. +Do not silently unwrap agents, migrate their state, or retain a combined-host compatibility shim. + +Workflow hosts expose checkpoint storage rather than agent-session storage or agent history settings. +Both Invocations hosts inherit directly from the SDK host and own their mode-specific construction, +parsing, state, and execution. Do not retain a private combined host or replace it with a mode-switching +base or mixin. Share small identity, wire, and streaming-cleanup helpers rather than duplicating protocol +logic. Group each protocol into agent, workflow, and shared modules, +with cross-protocol request contracts and factory resolution in a central shared module. + +Keep all hosts in one package. Responses is the initial stable-surface target; Invocations and Toolbox +remain experimental. Separate classes do not require separate distribution packages. + +### Request adaptation and construction + +Use `parse_input` for custom input adaptation across both protocols. Agent parsers return `InvocationRun`; +workflow parsers return `WorkflowTurn` containing either typed start input or complete pending replies. +Parsing does not replace the host's checkpoint and pending-authority validation. +Use `ResponsesRequest` for the Responses wrapper, including factory and option-hook inputs. +Do not retain aliases for the former parser keywords or request-type name. + +Agent hosts accept an instance or a sync/async factory implementing a named-request callable protocol. +Factories must declare a required `request` parameter and are called as `factory(request=request)`. +Additional parameters may have defaults; the factory may ignore the request. Reject zero-argument, +positional-only request, and implicit `**kwargs`-only factories rather than guessing which parameter +represents request context. Validate the signature before invocation. Never retry a factory-body +exception with a different call shape: that could duplicate side effects. + +Responses agent factories receive trusted scope, current-turn input, and translated caller options +before `prepare_options`. Invocations agent factories receive the Starlette `Request` after parsing +and option preparation. Preserve these protocol-specific timings rather than forcing identical signatures +or inventing a shared options-precedence policy. + +Native workflow factories are request-aware and must allocate fresh workflows, executors, agents, +clients, providers, and mutable tools. A new graph wrapper around shared executors is not isolation. +Reject known shared execution resources rather than cloning them. This guard cannot prove that arbitrary +application globals are isolated. Keep workflow names and executor IDs stable for checkpoint restoration. + +Construct tool objects inside the workflow factory as the default. Module-level `@tool` decoration +constructs one shared wrapper with its own lifetime counters and any captured resources. The current +ownership guard rejects reused function-tool objects without attempting to infer function purity. +A function may be defined elsewhere and wrapped inside the factory, but developers remain responsible +for captured or externally shared state. Fresh wrappers do not clone that state or establish isolation. + +Close factory-owned context-managed agents for the request, including failure and cancellation cleanup. +Applications own cleanup of other resources they create; caller-owned shared resources must not be closed. +State needed across requests belongs in supported stores, not request-owned instance fields. + +### Persistence and history ownership + +Regular agents continue through agent-session storage. Native workflows continue through their exact +scoped checkpoints, including pending-reply authority. Neither execution model converts the other's state. +Native workflow agents use checkpoint-owned history, not private inference-service continuation. + +Keep three storage decisions distinct: + +- The wire-level `store` flag controls caller-facing Responses persistence. +- The constructor's `response_store` selects the backend for that persistence. +- `history_source` selects who supplies prior messages to the model. + +Use `history_source="host_managed"` by default, with `"agent_managed"` and `"inference_service"` +as explicit alternatives. `agent_managed` permits developer-owned history providers or downstream +service storage; it is not restricted to `HistoryProvider`. Remove the constructor `store=` alias, +including acceptance of `None`, without changing the wire flag or downstream storage controls. + +### Background execution and recovery + +Treat outer background execution, inner inference-service background execution, and recovery permission +as separate controls. Use `background_source="host_managed"` by default. Retain `"inference_service"` +for supported regular agents with inference-service history and outer-background passthrough. +Reject hidden provider background under an outer foreground request. + +Outer workflow background must not enable background on constituent agents. Native inner provider +background is unsupported. Do not introduce a per-agent workflow-host background switch. +Recovery permission neither selects an execution mode nor guarantees exactly-once external side effects. + +### Supported host settings + +Expose supported first-class settings and construct SDK options internally, without another framework +options wrapper. Validate framework-owned settings and their applicability before side effects. +Remove variadic constructor `**kwargs` from all four public hosts. Unknown top-level names fail through +Python argument binding, including removed names supplied as `None`. + +Expose `agent_server_kwargs: Mapping[str, Any] | None = None` as a separate, caller-owned SDK boundary. +Expand its entries only into the upstream SDK constructor, without inspecting, validating, filtering, +or mutating the contents. Let Python and the SDK report invalid arguments and duplicate keyword +collisions; do not add a reserved-key blacklist or a custom merge policy. This does not remove validation +of framework parameters, caller model options, hooks, or execution policies. + +| Setting | Contract | +| --- | --- | +| `log_level=None` | Available on all four hosts as `Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] \| None`, forwarded unchanged to the SDK constructor. `None` preserves the SDK default. | +| `max_history_items: int \| None = None` | `None` supplies no framework override and inherits the installed supported SDK's default. Positive caps count prior history items and apply only to regular host-managed agent history. | +| `sse_keep_alive_interval_seconds: int \| None = None` | Available on all four framed-SSE hosts. Explicit values must be positive; zero and negative values are invalid. `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise disables heartbeats. No separate force-disable switch. | +| `shutdown_grace_period_seconds: int = 10` | Shutdown waiting, not a per-request timeout. Also document the upstream reuse for DELETE waiting on response persistence. | +| `resilient_background: bool = False` | Permit recovery of eligible background work without selecting background execution. | + +Do not interpret a history limit of `None` as unlimited, map it to `-1`, or raise the dependency floor +to obtain a particular SDK default. History caps do not limit tokens, turns, retention, or checkpoints. +Heartbeats are not model or request timeouts. + +Configure inference defaults on agents and clients, not through host `default_model`. Do not expose +host span callbacks, server-version stamping, or steering settings. Retain framework observability +and keep steering disabled. Removing host defaults does not redefine caller-supplied request models +or other request-option precedence. + +### Approvals and consent + +The native agent owns approval tracking, pending snapshots, policy decisions, binding, consumption, +and execution in its `AgentSession`. Hosting persists the complete session rather than implementing a +second approval store or a Foundry-specific approval dictionary in session state. +Native workflow pending authority remains checkpoint-owned. + +AgentServer's wire IDs differ from native approval IDs. Retain only transport correlation in the +response item's internal metadata; it is not executable authority. The native session must still +contain the matching pending request. Save that session before publishing a resumable approval. +Remove the separate approval-store configuration and exports without silently migrating old pending work. + +This migration preserves Core's existing session-level guarantees. Durable single-consumption across +workers and independently restored old snapshots requires separate native session-store coordination. +Do not implement that guarantee in a hosting-specific approval manager. + +The intended HTTP contract rejects incomplete hosted MCP approval sets before streaming, response headers, +`response.created`, or protected dispatch, without consuming pending authority. A complete batch may mix +approvals and rejections. Exact pre-header HTTP 400 behavior remains blocked on a suitable public SDK +validation hook under #9537; the native approval migration does not claim to implement that requirement. + +Represent invocation-time OAuth consent as structured consent with incomplete response status. +Pair each OAuth-interrupted invocation with one terminal result and preserve every authorized batch result. +OAuth consent does not suspend sibling calls: let the batch finish under its configured concurrency, +then return all consent requests with the complete results. Existing pre-execution approval gating +remains unchanged; eager execution before approval is separate vNext work tracked in #10334. +Require `store=true` so continuation can retain the required state; +do not support ephemeral consent under `store=false`. +Preserve results rather than cancelling or automatically replaying completed calls. +For inference-service history, reconcile a fully terminal batch before publishing consent and retain the accepted +continuation even when the service's reconciliation output is rejected. When another call still requires approval +or other input, publish both requests and retain completed results and unsent caller input in the native pending +batch until that call is resolved. Submit the real results together on continuation without replaying completed +calls or fabricating a terminal result for a pending call. +This behavior depends on the coordinated native Core change; select and validate the released Core +dependency floor during release preparation rather than adding a hosting-only compatibility fallback. + +### Unresolved boundaries + +Do not infer a single-target restriction or a multi-target router without a documented Foundry platform +contract. Request `model` is not implicitly an agent selector. + +Broader option precedence and validation details require separate design. The host split does not +authorize removing existing hooks or applying workflow-only assumptions to regular agents. + +Keep current dependency requirements and the steering guard until published-wheel compatibility and +concurrent queue-overflow behavior are verified. An upstream merge alone is not release evidence. +These decisions do not redesign Core, concurrency, checkpoints, retries, or external exactly-once execution. + +## Consequences + +- Public signatures identify execution models and their state owners. +- Shared internals avoid duplicate engines without obscuring public contracts. +- Request-aware factories support identity-dependent resources but require deliberate isolation and cleanup. +- The API migration breaks combined-host construction, wrapper hosting, unnamed or zero-argument agent factories, old parser/request names, source literals, and the storage-backend alias. +- SDK constructor arguments move from arbitrary top-level keywords to the explicit, unvalidated `agent_server_kwargs` mapping. The public SDK options bag is replaced by supported first-class settings. +- Settings, authority, and consent changes require their own implementation and correctness validation; recording a decision does not establish that it has shipped. + +## Validation + +Cover public imports and signatures, invalid construction before SDK side effects, sync/async factory +invocation, failure/cancellation cleanup, and rejection of shared workflow resources. +Exercise streaming and non-streaming execution, session/checkpoint continuation, pending replies, +and model-free process crash/recovery. Validate maintained samples against public exports. +Offline checks do not establish live Foundry deployment compatibility. diff --git a/docs/specs/004-python-function-calling-loop.md b/docs/specs/004-python-function-calling-loop.md index f755951845d..697ae435808 100644 --- a/docs/specs/004-python-function-calling-loop.md +++ b/docs/specs/004-python-function-calling-loop.md @@ -397,6 +397,23 @@ that manually replay messages own the equivalent rule: do not resend an approval cancellation remains cancellation rather than becoming this provider signal. - Every actionable local `function_call` produces exactly one terminal `function_result`, unless execution pauses for a new user-input request or the run is aborted by a fail-closed error. +- A tool-raised OAuth consent request is a terminally paired interruption, not an unresolved function call. + Return one consent-required result for its call and preserve the ordered results of the complete authorized batch. + OAuth consent alone does not suspend or cancel sibling calls, including MCP calls waiting for admission. Let + eligible calls finish under the configured concurrency, collect all consent requests, and count attempted calls + normally. Do not automatically replay protected operations. Existing pre-execution approval gating is unchanged, + and provider-issued consent keeps its separate ownership. + With service-owned history and a fully terminal batch, submit the complete locally produced result batch once + in the foreground with tools disabled before publishing the consent pause, advance continuation state, and account + for usage without exposing the model's reconciliation prose. If another call still requires approval or other user + input, return the requests together and retain completed results and unsent caller input in the native pending + batch instead. Submit all real results once every pending call is resolved; do not reconcile partial outputs, + fabricate a result for a pending call, or replay a completed sibling. Manual-history callers retain the equivalent + pending transcript. This applies to ordinary execution and approved-tool replay in both response modes. + Unfinished reconciliation responses and failures propagate; an accepted service continuation remains advanced if + its returned output is rejected. Cancellation or an ambiguous transport failure does not authorize an automatic retry. + Approval-time reconciliation preserves new caller input alongside normalized results without sending OAuth + actions or replaying resolved local approval controls to the service. - An ordinary exception raised by function middleware or a tool body becomes one terminal error `function_result` and the loop continues. `MiddlewareFailure` is the execution-time fail-closed escape: it is never converted into a tool result, the in-flight parallel batch is cancelled, no further tool call starts, no further model turn is @@ -641,8 +658,11 @@ that manually replay messages own the equivalent rule: do not resend an approval | Mixed approved/rejected batch | Every call gets one correctly correlated terminal result. | `packages/core/tests/core/test_function_invocation_logic.py::test_rejected_approval` | | Persisted approval replay | Resume executes with the prior call available. | `test_persisted_approval_messages_replay_correctly` | | Hosted approval pass-through | Hosted requests/responses are bound to the recorded provider request and are not processed as local calls. | `test_hosted_tool_approval_response`, `test_hosted_mcp_approval_response_passthrough`, `test_session_approval_binding_reconstructs_hosted_response`, `test_mixed_local_and_hosted_approval_flow` | +| Foundry native approval round trip | HTTP approval and rejection resume the issuing serialized native session in streaming and non-streaming modes; completed decisions cannot execute again. | `packages/foundry_hosting/tests/test_responses.py::TestFunctionApprovalRoundTrip::test_native_agent_consumes_serialized_session_approval`, `test_native_approval_batch_waits_for_all_decisions` | +| Foundry native approval publication | Only requests matching native pending state are published, after successful native finalization and session persistence; failures publish no approval item. | `packages/foundry_hosting/tests/test_responses.py::TestFunctionApprovalRoundTrip::test_non_streaming_emits_mcp_approval_request_and_persists_native_session`, `test_streaming_emits_mcp_approval_request_and_persists_native_session`, `test_custom_agent_cannot_publish_approval_without_matching_native_state`, `test_failed_native_finalization_does_not_publish_approval`, `test_session_save_failure_does_not_publish_resumable_approval` | +| Foundry deferred approval output | Output order is preserved across finalization; count and serialized-byte limits fail explicitly without publishing the buffered approval, and an overflowing producer is closed in both response modes. | `packages/foundry_hosting/tests/test_responses.py::TestFunctionApprovalRoundTrip::test_deferred_approval_preserves_streamed_output_order`, `test_deferred_approval_output_has_count_and_byte_limits`, `test_deferred_approval_output_enforces_default_limits`, `test_deferred_approval_limit_closes_an_unending_stream` | | Agent-tool child approval | A child `ToolApprovalMiddleware` can auto-approve runtime tool requests inside one delegated invocation; unresolved child approvals execute nothing and do not enter the caller's approval state or dispatch a same-named parent tool, including with propagated application state. Mixed batches preserve non-approval user-input requests. Shared parent and child state rejects overlapping `ToolApprovalMiddleware.source_id` values before running the child; parent approval state, including state of middleware passed to a single parent run, is neither visible nor mutable inside the child (a child cannot add standing rules or forge a decision for a pending request), and custom child approval queues do not leak into later delegations. | `packages/core/tests/core/test_agents.py::test_chat_agent_as_tool_auto_approves_child_tool_with_middleware`, `test_chat_agent_as_tool_fails_closed_for_unresolved_child_approval`, `test_chat_agent_as_tool_child_approval_does_not_dispatch_same_named_parent_tool`, `test_chat_agent_as_tool_preserves_non_approval_requests_from_mixed_child_batch`, `test_chat_agent_as_tool_shared_session_requires_distinct_tool_approval_source_ids`, `test_chat_agent_as_tool_isolates_custom_parent_approval_state`, `test_chat_agent_as_tool_isolates_run_level_parent_approval_state`, `test_chat_agent_as_tool_run_level_approval_state_cannot_authorize_pending_request`, `test_chat_agent_as_tool_run_level_approval_ownership_is_per_invocation`, `test_chat_agent_as_tool_approved_delegation_does_not_confuse_framework_approval_state`, `test_chat_agent_as_tool_does_not_restore_custom_approval_queue_on_fresh_delegation` | -| Approval-time user input | Every user-input request from one approved execution returns in order with assistant role and no extra model call; the execution consumes one call-budget unit. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_all_user_input_requests_without_another_model_call`, `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_user_input_counts_toward_function_call_budget` | +| Approval-time user input | Every user-input request from one approved execution returns in order with assistant role; OAuth also retains its paired tool-role result. Local history needs no extra model call; service-owned OAuth reconciliation follows the contract below. The execution consumes one call-budget unit. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_all_user_input_requests_without_another_model_call`, `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_user_input_counts_toward_function_call_budget` | | Mixed terminal result and follow-up input | Completed siblings remain tool-role while only follow-up input requests use assistant-role messages/updates. | `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_separates_terminal_results_from_follow_up_requests`, `packages/openai/tests/openai/test_openai_chat_completion_client.py::test_mixed_approval_resume_roles_serialize_function_result_as_tool`, `packages/core/tests/core/test_harness_tool_approval.py::test_dynamic_policy_approval_partitions_safe_sibling_result_roles` | | Approval-time middleware termination | Terminal result returns with no extra model call in either response mode. | `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_honors_middleware_termination` | | Approval re-entry after iteration budget | Pending approved calls resolve once even when prior model calls consumed `max_iterations`. | `packages/core/tests/core/test_harness_tool_approval.py::test_auto_approval_resolves_after_iteration_budget_is_exhausted` | @@ -684,6 +704,8 @@ that manually replay messages own the equivalent rule: do not resend an approval | Local approval without authoritative session | A local approval response carried by any message role executes nothing when no session recorded its request, whether the request is fabricated, absent, answered with tampered arguments, or repeated as several responses sharing one approval id. Verified for both streaming and non-streaming. | `test_local_approval_response_without_authoritative_session_does_not_execute`, `test_duplicate_approval_id_reclassification_does_not_execute_without_session` | | Filtering precedes batch validation | An unbound local response is removed before stateless mixed-batch completeness is enforced, so it is never counted as an answer and the remaining history is validated on its own merits. | `test_unbound_local_approval_response_is_filtered_before_mixed_batch_validation` | | Session-bound approval round trip | The same response executes once when the issuing session is supplied on the resuming run. | `test_local_approval_response_executes_with_authoritative_session` | +| Foundry wire/session isolation | A wire approval cannot bind to a different native session or to a later request that reuses the native ID with a changed server, tool, or arguments; rejected aliases leave pending authority intact in both response modes. | `packages/foundry_hosting/tests/test_responses.py::TestFunctionApprovalRoundTrip::test_wire_correlation_cannot_cross_native_sessions`, `test_stale_wire_approval_cannot_bind_to_reused_native_id` | +| Foundry wire argument serialization | Equivalent JSON argument representations still bind to the native snapshot without coercing booleans to numbers or rounding distinct numeric values. | `packages/foundry_hosting/tests/test_responses.py::TestFunctionApprovalRoundTrip::test_wire_approval_accepts_equivalent_argument_serialization`, `test_stale_wire_approval_cannot_bind_to_reused_native_id` | | Hosted approval pass-through | Provider-issued approvals still reach the provider untouched without a session. | `test_hosted_approval_response_passes_through_without_session` | | Settled replay without a session | A completed approval round replays as history: it does not re-execute, does not warn, and does not fail a completed mixed batch. | `test_settled_approval_response_replays_without_session_or_warning`, `test_completed_split_stateless_mixed_batch_is_inert_on_later_turn` | | Binding opt-out | `disable_approval_response_binding` restores the previous unbound pass-through. | `test_disable_approval_response_binding_restores_unbound_behavior` | @@ -731,6 +753,9 @@ that manually replay messages own the equivalent rule: do not resend an approval | Pre-validation middleware repair | Schema-compatible calls retain normalized middleware arguments without duplicate validation. Prepared-value reuse does not require validator outputs to be copyable, and unchanged NaNs remain stable. Identity-only opaque values may be reused for ordinary execution, but security authority rejects them before downstream middleware or the tool body because in-place mutation cannot be detected safely. When provisional validation fails, function middleware observes raw parsed arguments, may repair them before final validation, and the body receives normalized values; short-circuiting skips final validation and execution. Repair after security middleware fails closed using recursive type-aware, float-bit-exact comparison, including invalid and short-circuited mutations. Security inspects exact normalized values, and validation errors after hidden-value resolution do not disclose resolved values or mapping keys, including validator `TypeError` paths. | `test_function_middleware_keeps_normalized_arguments_for_valid_calls`, `test_prepared_arguments_support_noncopyable_validator_output`, `test_nan_prepared_and_approval_snapshots_are_stable`, `test_function_middleware_repairs_raw_arguments_before_validation`, `test_function_middleware_can_short_circuit_before_argument_validation`, `test_invalid_arguments_produced_by_middleware_keep_argument_error_contract`, `packages/core/tests/test_security.py::TestVariableArgumentPolicy::test_security_rejects_opaque_mutable_validator_output`, `test_argument_mutation_after_security_middleware_fails_closed`, `test_security_snapshot_accepts_unchanged_nan`, `test_argument_mutation_after_security_short_circuit_fails_closed`, `test_security_policy_observes_custom_validator_transform_once`, `test_hidden_argument_validation_error_does_not_disclose_resolved_value`, `test_hidden_mapping_key_is_not_disclosed_by_validation_error`, `test_hidden_value_is_not_disclosed_by_validator_type_error`, `test_hidden_argument_can_be_normalized_after_security_check` | | Approved middleware repair | Approval binds to the normalized middleware-entry representation, so ordinary Pydantic coercion still completes in one approval round. A changed approval-bound call executes zero times under the old grant, returns a persisted occurrence-bound replacement request in both response modes, and executes once only after the replacement is approved. Recursive type-aware, float-bit-exact comparison treats booleans and numbers, and positive and negative zero, as distinct while keeping unchanged NaNs stable; set elements and mapping entries are counted, so removing one of several NaNs is detected. Opaque mutable normalized values fail closed before approval authority is established. The replacement holds JSON-native arguments that survive session serialization, and a repair whose JSON form would prepare to different values fails closed. The same replacement rule applies when middleware short-circuits instead of calling the tool. Security expansion preserves approval-visible placeholders. | `test_approved_coercing_arguments_execute_without_replacement`, `test_approved_argument_repair_requires_replacement_approval`, `test_approved_argument_repair_short_circuit_requires_replacement_approval`, `test_approval_snapshot_distinguishes_exact_values`, `test_approval_rejects_opaque_mutable_validator_output`, `test_approval_detects_changes_to_converted_values`, `test_approval_change_that_json_cannot_represent_fails_closed`, `test_argument_tokens_count_items_that_share_a_token`, `test_approved_repair_replacement_round_trips_through_json`, `packages/core/tests/test_security.py::TestVariableArgumentPolicy::test_hidden_argument_resolution_does_not_require_reapproval` | | Approved success | Successful approved execution returns one result. | `test_approved_function_call_successful_execution` | +| Invocation-time OAuth consent | Ordinary and approved batches finish under configured concurrency, retain one terminal result per attempted call, collect all consent requests in model order, preserve successful and failed sibling results, and count every attempt normally. Consent does not block queued MCP siblings or change pre-execution approval gating. | `packages/core/tests/core/test_function_invocation_logic.py::test_oauth_consent_preserves_all_batch_results`, `test_oauth_consent_preserves_already_dispatched_sibling`, `test_oauth_consent_supplied_terminal_result_stays_single_and_immutable`, `packages/foundry_hosting/tests/test_toolbox.py::test_toolbox_oauth_consent_allows_an_internally_queued_mcp_dispatch`, `test_toolbox_oauth_consent_remains_distinct_from_native_policy_approval`, `test_toolbox_oauth_consent_collects_requests_and_preserves_sibling_results` | +| OAuth service-history reconciliation | Submit a fully terminal batch once before exposing consent, advance the service continuation in streaming and final output, and preserve the accepted continuation if new service tool calls are rejected. A mixed pending batch retains completed results, unsent input, and its budget until the remaining approval/input is resolved, then submits all real results without replay. Failures and cancellation never publish a successful consent pause or retry settlement. Provider-owned consent is not locally settled again. | `packages/core/tests/core/test_function_invocation_logic.py::test_oauth_consent_reconciles_real_results_and_advances_service_history`, `test_oauth_mixed_pause_retains_partial_input_and_budget`, `test_oauth_consent_settlement_failure_withholds_user_input`, `test_provider_oauth_consent_is_not_locally_settled`, `packages/foundry_hosting/tests/test_toolbox.py::test_toolbox_oauth_consent_pairs_results_and_continues`, `test_toolbox_oauth_consent_preserves_a_mixed_pending_batch`, `test_toolbox_oauth_consent_reconciliation_failure_is_not_an_incomplete_response`, `test_toolbox_oauth_consent_background_run_reconciles_in_foreground` | +| Foundry Toolbox OAuth boundary | Only validated structured Toolbox consent becomes a consent action. Generic MCP and malformed failures keep their error contract, links obey host policy, unstored consent fails explicitly without persistence, and stored response/conversation continuation does not replay completed siblings. | `packages/foundry_hosting/tests/test_toolbox.py::test_toolbox_oauth_consent_does_not_classify_unvalidated_errors`, `test_generic_mcp_does_not_translate_toolbox_consent`, `test_toolbox_oauth_consent_origin_allowlist_is_enforced`, `test_toolbox_oauth_consent_rejects_unstored_requests`, `test_toolbox_oauth_consent_pairs_results_and_continues` | | Consecutive error cap | Error threshold stops repeated failures, submits collected results, and makes only the required final no-tool model call. | `test_function_invocation_config_max_consecutive_errors`, `test_streaming_function_invocation_config_max_consecutive_errors`, `test_approval_resume_error_limit_forces_final_no_tool_response` | | Unknown call handling | Configured false returns an error result; configured true raises before any call executes. A service-managed continuation settles every call in the aborted batch and advances to the settlement response before the `KeyError` propagates; without one, including local per-service-call history persistence, no extra request is made. | `test_function_invocation_config_terminate_on_unknown_calls_false`, `test_function_invocation_config_terminate_on_unknown_calls_true`, streaming equivalents, `test_fatal_unknown_settles_service_conversation`, `test_fatal_unknown_with_local_history_makes_no_settlement_request` | | Middleware termination | Normal non-approval loop stops without a second model call. | `test_terminate_loop_single_function_call`, `test_terminate_loop_multiple_function_calls_one_terminates`, `test_terminate_loop_streaming_single_function_call` | @@ -758,6 +783,7 @@ that manually replay messages own the equivalent rule: do not resend an approval | Invalidated per-service-call persistence | An invalidated service response, its partial calls, and any nonexistent results are not success-persisted, and invalid content does not advance session continuation state. | `packages/core/tests/core/test_function_invocation_logic.py::test_invalidated_response_is_not_persisted` | | Streaming message injection with per-service-call persistence | A streaming response rebuilt from updates mirrors the inner conversation id exactly, including clearing it, and keeps its internal marker, so the next iteration appends only the latest message rather than replaying the whole turn on top of provider-held history, and never persists a conversation id from an earlier injected service call. | `packages/core/tests/core/test_middleware_with_chat.py::TestChatMiddleware::test_message_injection_middleware_streaming_preserves_inner_continuation_state`, `test_message_injection_middleware_streaming_keeps_service_conversation_id_external`, `test_message_injection_middleware_streaming_clears_conversation_id_when_final_call_has_none`, `test_message_injection_middleware_conversation_id_matches_across_streaming_modes`, `packages/core/tests/core/test_harness_agent.py::test_streaming_harness_tool_call_does_not_duplicate_transcript` | | Service-side approval decision | Stored hosted request is skipped; the current approved or rejected hosted response is sent, while local approval controls are omitted from provider input. | `packages/openai/tests/openai/test_openai_chat_client.py::test_prepare_messages_strips_approval_request_but_keeps_response_under_storage`, `test_prepare_messages_drops_local_approval_controls` | +| Foundry history-owned approval replay | Host-managed stateless input restores the pending hosted request even with capped wire history; agent-managed or service-side history receives no duplicate, while both response modes serialize exactly one current provider decision. | `packages/foundry_hosting/tests/test_responses.py::TestFunctionApprovalRoundTrip::test_hosted_approval_replays_only_with_the_owner_of_history`, `test_native_agent_consumes_serialized_session_approval` | | OpenAI approval serialization | Hosted approval id and decision serialize to `mcp_approval_response`; local approvals remain in-process. | `test_prepare_message_for_openai_with_function_approval_response`, `test_prepare_content_for_opentool_approval_response`, `test_function_approval_response_with_mcp_tool_call` | | OpenAI end-to-end hosted approval | Hosted request parses, response sends, and continuation completes. | `test_end_to_end_mcp_approval_flow` | | Stored function call/result | Service-side storage drops server-issued calls but keeps new outputs. | `test_prepare_options_with_conversation_id_strips_server_issued_items`, `test_prepare_messages_for_openai_full_conversation_with_reasoning` | @@ -769,7 +795,7 @@ that manually replay messages own the equivalent rule: do not resend an approval | DevUI conversation-bound approval ownership | A pending function approval is bound to the conversation that originated it. A response from another conversation is rejected without consuming the pending approval, and identical request IDs in different conversations remain independent. | `packages/devui/tests/devui/test_approval_validation.py::test_approval_request_must_be_bound_to_originating_conversation`, `test_same_approval_request_id_is_scoped_per_conversation` | | AG-UI approval result event | Approved result emits once with content and persists in snapshot. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_approved_call_emits_one_live_result_under_original_identity`, `packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_agent_approval_resume_persists_replayable_tool_results`, `test_endpoint_agent_approval_replayed_resume_entry_reprojects_retained_result` | | AG-UI rejection/mixed decision | Transport emits only the events defined for approved and rejected calls without duplicates. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_rejected_call_does_not_execute_or_emit_live_result`, `test_mixed_batch_preserves_approved_result_identity_and_order`, `packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_agent_approval_rejection_releases_already_approved_sibling` | -| AG-UI approval-time follow-up | The full grouped user-input pause remains in message history and emits no synthetic `TOOL_CALL_RESULT`. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_approval_follow_up_group_remains_in_history_without_live_tool_result` | +| AG-UI approval-time follow-up | The full grouped user-input pause remains in message history. OAuth emits its single paired terminal `TOOL_CALL_RESULT`; other follow-up input does not fabricate a terminal result. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_approval_follow_up_group_preserves_terminal_result_contract` | | AG-UI approval execution failure | A grouped executor failure becomes one deterministic terminal error result for the approved call. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_approval_execution_failure_emits_one_terminal_error_result` | | AG-UI no-approval path | Ordinary tool results do not gain an extra approval result event. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_no_approval_path_emits_no_approval_specific_duplicate_result` | | AG-UI client-tool request isolation | Client tool declarations are validated before use and remain request-scoped; a rejected collision or earlier successful request cannot change a later request's server-tool execution. | `packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_failed_client_tool_collision_does_not_affect_next_request`, `test_endpoint_client_tools_do_not_persist_into_next_request` | diff --git a/python/PACKAGE_STATUS.md b/python/PACKAGE_STATUS.md index 424b4ef44ec..4e25ff17bd6 100644 --- a/python/PACKAGE_STATUS.md +++ b/python/PACKAGE_STATUS.md @@ -156,7 +156,7 @@ listed below. - `agent-framework-core`: `SessionStore` and `FileSessionStore` from `agent_framework/_sessions.py` - `agent-framework-foundry-hosting`: `FoundryAgentSessionStore` from - `agent_framework_foundry_hosting/_state_store.py` + `agent_framework_foundry_hosting/_shared.py` #### `TO_PROMPT_AGENT` diff --git a/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py b/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py index 32c4e5065cc..9489271e56b 100644 --- a/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py +++ b/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py @@ -7,6 +7,7 @@ import json from typing import Any +import pytest from ag_ui.core import ToolCallResultEvent from agent_framework import AgentResponseUpdate, Content, FunctionInvocationContext, FunctionTool from agent_framework.exceptions import UserInputRequiredException @@ -262,31 +263,41 @@ async def test_mixed_batch_preserves_approved_result_identity_and_order() -> Non assert [(event.tool_call_id, event.content) for event in results] == [("call-seattle", "Sunny in Seattle")] -async def test_approval_follow_up_group_remains_in_history_without_live_tool_result() -> None: - """Approval-time user-input requests remain grouped without becoming terminal tool results.""" +@pytest.mark.parametrize("oauth", [False, True], ids=["user-input", "oauth"]) +async def test_approval_follow_up_group_preserves_terminal_result_contract(oauth: bool) -> None: + """OAuth has a paired terminal result; other follow-up input remains pending.""" + prompts = ( + [ + Content.from_oauth_consent_request(consent_link="https://example.com/first"), + Content.from_oauth_consent_request(consent_link="https://example.com/second"), + ] + if oauth + else [Content("text", text="Please provide more input.", user_input_request=True)] + ) def request_consent() -> str: - raise UserInputRequiredException( - contents=[ - Content.from_oauth_consent_request(consent_link="https://example.com/first"), - Content.from_oauth_consent_request(consent_link="https://example.com/second"), - ] - ) + raise UserInputRequiredException(contents=prompts) events, agent = await _run_custom_approval( FunctionTool(name="request_consent", description="Request consent", func=request_consent) ) - assert not [event for event in events if getattr(event, "type", None) == "TOOL_CALL_RESULT"] + results = [event for event in events if isinstance(event, ToolCallResultEvent)] + if oauth: + assert len(results) == 1 + assert results[0].tool_call_id == "call-custom" + assert "OAuth consent is required" in results[0].content + assert "https://" not in results[0].content + else: + assert not results requests = [ content for message in agent.messages_received for content in message.contents if isinstance(content, Content) and content.user_input_request ] - assert [request.consent_link for request in requests] == [ - "https://example.com/first", - "https://example.com/second", + assert [(request.type, request.consent_link, request.text) for request in requests] == [ + (prompt.type, prompt.consent_link, prompt.text) for prompt in prompts ] diff --git a/python/packages/core/agent_framework/_mcp.py b/python/packages/core/agent_framework/_mcp.py index 41e37a0d9ff..3dbfca07fdc 100644 --- a/python/packages/core/agent_framework/_mcp.py +++ b/python/packages/core/agent_framework/_mcp.py @@ -50,7 +50,7 @@ Content, Message, ) -from .exceptions import ToolException, ToolExecutionException +from .exceptions import ToolException, ToolExecutionException, UserInputRequiredException from .observability import ( OtelAttr, create_mcp_client_span, @@ -3742,6 +3742,8 @@ async def _call_tool_with_retries( span_error_recorded = True raise _mcp_tool_execution_exception(text or str(parsed)) return parser(result) + except UserInputRequiredException: + raise except ToolExecutionException as tool_ex: if span.is_recording() and not span_error_recorded: set_mcp_span_error(span, type(tool_ex).__name__, str(tool_ex)) diff --git a/python/packages/core/agent_framework/_tools.py b/python/packages/core/agent_framework/_tools.py index 9a1f142cfb7..47da0cd7fe3 100644 --- a/python/packages/core/agent_framework/_tools.py +++ b/python/packages/core/agent_framework/_tools.py @@ -2645,9 +2645,33 @@ async def _execute_single_function_call( except UserInputRequiredException as exc: source_function_call = _underlying_function_call(function_call) call_id = source_function_call.call_id - propagated_contents = [item for item in exc.contents if isinstance(item, Content)] if exc.contents else [] + propagated_contents = ( + [copy.copy(item) for item in exc.contents if isinstance(item, Content)] if exc.contents else [] + ) + if any(item.type == "oauth_consent_request" for item in propagated_contents): + terminal_results = [item for item in propagated_contents if item.type == "function_result"] + if len(terminal_results) > 1: + raise ValueError( + "An OAuth consent interruption must contain at most one terminal function result." + ) from exc + if not terminal_results: + propagated_contents.insert( + 0, + Content.from_function_result( + call_id=call_id, # type: ignore[arg-type] + id=source_function_call.id, + result=( + "Error: OAuth consent is required. This tool invocation did not complete. " + "The user must authorize access before this tool can be invoked in a later request." + ), + exception="OAuthConsentRequired", + additional_properties=source_function_call.additional_properties, + ), + ) for item in propagated_contents: item.call_id = call_id + if item.type == "function_result": + item.id = source_function_call.id if not item.id: item.id = source_function_call.id or call_id if propagated_contents: @@ -3543,8 +3567,9 @@ def _store_pending_mixed_pause_batch( *, deferred_function_calls: Sequence[Content] = (), function_call_order: Sequence[Mapping[str, str | None]] | None = None, + deferred_messages: Sequence[Message] = (), ) -> None: - """Persist the active ordered pause batch and any deferred local siblings.""" + """Persist the active ordered pause batch, completed results, and deferred local siblings.""" state = _get_tool_approval_state(invocation_session) if state is None: return @@ -3553,21 +3578,34 @@ def _store_pending_mixed_pause_batch( kinds: set[str] = set() for group in pause_groups: for content in group: + if content.type == "function_result": + kinds.add("result") + items.append({"kind": "result", "response": content.to_dict()}) + continue if content.type == "function_approval_request": kind = "approval" - elif content.type == "function_call" and content.user_input_request: + elif content.user_input_request and content.type != "oauth_consent_request": kind = "host" else: continue kinds.add(kind) items.append({"kind": kind, "request": content.to_dict()}) - if kinds == {"approval", "host"} or (kinds == {"host"} and deferred_function_calls): + if ( + kinds == {"approval", "host"} + or (kinds == {"host"} and deferred_function_calls) + or ("result" in kinds and bool(kinds & {"approval", "host"})) + ): batch: dict[str, Any] = {"items": items} if deferred_function_calls: batch[_DEFERRED_FUNCTION_CALLS_KEY] = [function_call.to_dict() for function_call in deferred_function_calls] if function_call_order is not None: batch[_FUNCTION_CALL_ORDER_KEY] = [dict(item) for item in function_call_order] + if deferred_messages: + retained_messages = _copy_messages_for_function_invocation(deferred_messages) + for message in retained_messages: + message.contents = [content for content in message.contents if content.type != "oauth_consent_request"] + batch["messages"] = [message.to_dict() for message in retained_messages if message.contents] state[_PENDING_MIXED_PAUSE_BATCH_KEY] = batch else: state.pop(_PENDING_MIXED_PAUSE_BATCH_KEY, None) @@ -3593,7 +3631,7 @@ def _match_mixed_pause_responses( host_items_by_occurrence: dict[str, int] = {} host_items_by_call: dict[str, list[int]] = {} for index, item in enumerate(items): - request = _content_from_state(item.get("request")) + request = _content_from_state(item.get("request", item.get("response"))) if request is None: continue if item.get("kind") == "approval": @@ -3607,7 +3645,7 @@ def _match_mixed_pause_responses( approval_items[identity] = index elif approval_items[identity] != index: approval_items[identity] = None - elif item.get("kind") == "host" and request.call_id is not None: + elif item.get("kind") in {"host", "result"} and request.call_id is not None: host_items_by_call.setdefault(request.call_id, []).append(index) if request.id is not None: host_items_by_occurrence[request.id] = index @@ -3645,7 +3683,11 @@ def _match_mixed_pause_responses( candidate = response if response.id is not None: item_index = host_items_by_occurrence.get(response.id) - request = _content_from_state(items[item_index].get("request")) if item_index is not None else None + request = ( + _content_from_state(items[item_index].get("request", items[item_index].get("response"))) + if item_index is not None + else None + ) if request is None or request.call_id != response.call_id: item_index = None else: @@ -3714,7 +3756,7 @@ def _match_mixed_pause_responses( if response is None: return matched_content_ids, True, [], set() ordered_responses.append(response) - if item.get("kind") == "host": + if item.get("kind") in {"host", "result"}: host_result_ids.add(id(response)) return matched_content_ids, False, ordered_responses, host_result_ids @@ -3741,13 +3783,23 @@ def _stage_pending_mixed_pause_responses( if not items: state.pop(_PENDING_MIXED_PAUSE_BATCH_KEY, None) return False, False, set() + raw_messages = cast(Mapping[str, Any], raw_batch).get("messages") + if isinstance(raw_messages, list): + messages[:0] = _copy_messages_for_function_invocation([ + raw_message if isinstance(raw_message, Message) else Message.from_dict(cast(dict[str, Any], raw_message)) + for raw_message in cast(list[Any], raw_messages) + ]) def bind_approval_response(response: Content) -> Content | None: - return _bind_approval_response_to_pending_request( + bound = _bind_approval_response_to_pending_request( response, invocation_session, consume=False, ) + if bound is not None: + # Binding runs again after staging; retain replacement-request identity until then. + bound.id = bound.additional_properties[_APPROVAL_REQUEST_ID_KEY] + return bound def attributed_staged_replay_item_index( content: Content, @@ -3772,7 +3824,7 @@ def attributed_staged_replay_item_index( candidate_state, ): continue - request = _content_from_state(item.get("request")) + request = _content_from_state(item.get("request", item.get("response"))) if request is None: continue if item.get("kind") == "approval" and candidate.type == "function_approval_response": @@ -3795,7 +3847,7 @@ def attributed_staged_replay_item_index( if request_ids.intersection(candidate_ids): return item_index elif ( - item.get("kind") == "host" + item.get("kind") in {"host", "result"} and candidate.type == "function_result" and candidate.call_id == request.call_id and (candidate.id is None or candidate.id == request.id) @@ -3806,9 +3858,8 @@ def attributed_staged_replay_item_index( host_requests = [ request for item in items - if item.get("kind") == "host" - and item.get("response") is None - and (request := _content_from_state(item.get("request"))) is not None + if (item.get("kind") == "host" and item.get("response") is None) or item.get("kind") == "result" + if (request := _content_from_state(item.get("request", item.get("response")))) is not None and request.call_id is not None ] approval_request_ids = { @@ -3925,12 +3976,18 @@ def attributed_staged_replay_item_index( messages[:] = filtered_messages updated_batch = dict(cast(Mapping[str, Any], raw_batch)) updated_batch["items"] = items + if incomplete and any(item.get("kind") == "result" for item in items): + updated_batch["messages"] = [message.to_dict() for message in messages] state[_PENDING_MIXED_PAUSE_BATCH_KEY] = updated_batch if incomplete: return True, False, set() - messages.append(Message(role="user", contents=ordered_responses)) + if any(item.get("kind") == "result" for item in items): + response_messages, _ = _messages_and_updates_for_terminal_contents(ordered_responses) + messages.extend(response_messages) + else: + messages.append(Message(role="user", contents=ordered_responses)) return False, True, host_result_ids @@ -4890,9 +4947,7 @@ def _clear_budget_state_from_session( if pending: return pending_pause_batch = tool_state.get(_PENDING_MIXED_PAUSE_BATCH_KEY) - if isinstance(pending_pause_batch, Mapping) and cast(Mapping[str, Any], pending_pause_batch).get( - _DEFERRED_FUNCTION_CALLS_KEY - ): + if isinstance(pending_pause_batch, Mapping): updated_batch = dict(cast(Mapping[str, Any], pending_pause_batch)) updated_batch[_DEFERRED_BUDGET_STATE_KEY] = _portable_budget_snapshot(budget_state) tool_state[_PENDING_MIXED_PAUSE_BATCH_KEY] = updated_batch @@ -5145,6 +5200,7 @@ def _handle_function_call_results( errors_in_a_row=errors_in_a_row, action="return", function_call_count=function_call_count, + response_messages=response_messages, streaming_updates=streaming_updates, ) @@ -5746,6 +5802,97 @@ def _update_function_invocation_continuation_state( ): session.service_session_id = conversation_id + async def _settle_oauth_consent_results( + self, + *, + super_get_response: Callable[..., Any], + response: ChatResponse[Any], + result_messages: Sequence[Message], + options: dict[str, Any], + request_kwargs: dict[str, Any], + compaction_strategy: CompactionStrategy | None, + tokenizer: TokenizerProtocol | None, + invocation_session: AgentSession | None, + input_messages: Sequence[Message] | None = None, + ) -> ChatResponse[Any] | None: + """Reconcile terminal OAuth batches and retain mixed pauses until their remaining calls resolve.""" + from ._sessions import is_local_history_conversation_id + from ._types import ChatResponse, Message + + if not any( + content.type == "oauth_consent_request" for message in result_messages for content in message.contents + ): + return None + continuation_id = response.conversation_id or cast("str | None", options.get("conversation_id")) + if continuation_id is None or is_local_history_conversation_id(continuation_id): + return None + results = [ + copy.copy(content) + for message in result_messages + for content in message.contents + if content.type == "function_result" + ] + if not results: + raise RuntimeError("OAuth consent reconciliation requires terminal function results.") + completed_calls = {(content.id, content.call_id) for content in results} + pending_calls: set[tuple[str | None, str | None]] = set() + pause_contents: list[Content] = [] + for message in result_messages: + for content in message.contents: + if content.type == "function_result": + pause_contents.append(content) + elif content.type == "function_approval_request" or ( + content.user_input_request and content.type != "oauth_consent_request" + ): + source_call = content.function_call or content + identity = (source_call.id, source_call.call_id) + if identity not in completed_calls and identity not in pending_calls: + pause_contents.append(content) + pending_calls.add(identity) + if pending_calls: + _store_pending_mixed_pause_batch( + invocation_session, [pause_contents], deferred_messages=input_messages or () + ) + return None + settlement_options = {**options, "conversation_id": continuation_id, "tool_choice": "none"} + if "background" in settlement_options: + settlement_options["background"] = False + settlement_messages = ( + _copy_messages_for_function_invocation(input_messages) + if input_messages is not None + else [Message(role="tool", contents=results)] + ) + for message in settlement_messages: + message.contents = [content for content in message.contents if content.type != "oauth_consent_request"] + settlement = await super_get_response( + messages=[message for message in settlement_messages if message.contents], + stream=False, + options=settlement_options, + compaction_strategy=compaction_strategy, + tokenizer=tokenizer, + client_kwargs=request_kwargs, + ) + if not isinstance(settlement, ChatResponse) or settlement.conversation_id is None: + raise RuntimeError("OAuth consent reconciliation did not return a service continuation ID.") + settlement = cast("ChatResponse[Any]", settlement) + # The service accepted the outputs. Preserve that advance even if its new output is invalid. + self._update_function_invocation_continuation_state( + request_kwargs, settlement, session=invocation_session, options=options + ) + response.conversation_id = settlement.conversation_id + if settlement.response_id is not None: + response.response_id = settlement.response_id + if settlement.continuation_token is not None: + raise RuntimeError("OAuth consent reconciliation returned an unfinished service response.") + if any( + content.type in {"function_call", "function_approval_request", "computer_tool_call", "shell_tool_call"} + or content.user_input_request + for message in settlement.messages + for content in message.contents + ): + raise RuntimeError("The service requested new tool execution or user input while settling OAuth consent.") + return settlement + async def _settle_dangling_service_function_calls( self, *, @@ -5920,6 +6067,19 @@ async def settle_approval_replay_calls(function_calls: Sequence[Content]) -> Non ) if approval_processing.action == "return": response = ChatResponse(messages=list(function_call_messages)) + settlement = await self._settle_oauth_consent_results( + super_get_response=super_get_response, + response=response, + result_messages=approval_processing.response_messages, + options=options, + request_kwargs=request_kwargs, + compaction_strategy=compaction_strategy, + tokenizer=tokenizer, + invocation_session=invocation_session, + input_messages=prepared_messages, + ) + if settlement is not None: + aggregated_usage = add_usage_details(aggregated_usage, settlement.usage_details) response.usage_details = aggregated_usage _clear_budget_state_from_session(invocation_session, budget_state) return _clear_internal_conversation_id(response) @@ -6010,6 +6170,18 @@ async def settle_approval_replay_calls(function_calls: Sequence[Content]) -> Non function_processing.function_call_count, ) if function_processing.action == "return": + settlement = await self._settle_oauth_consent_results( + super_get_response=super_get_response, + response=response, + result_messages=function_processing.response_messages, + options=options, + request_kwargs=request_kwargs, + compaction_strategy=compaction_strategy, + tokenizer=tokenizer, + invocation_session=invocation_session, + ) + if settlement is not None: + aggregated_usage = add_usage_details(aggregated_usage, settlement.usage_details) response.usage_details = aggregated_usage _prepend_function_call_messages(response, function_call_messages[:terminal_prefix_length]) _clear_budget_state_from_session(invocation_session, budget_state) @@ -6086,7 +6258,7 @@ async def _stream_response_with_function_invocation( ) -> AsyncIterable[ChatResponseUpdate]: """Run the streaming function invocation loop.""" from ._middleware import MiddlewareFailure - from ._types import ChatResponseUpdate, Content, add_usage_details + from ._types import ChatResponse, ChatResponseUpdate, Content, add_usage_details errors_in_a_row = 0 total_function_calls = int(budget_state.get("total_function_calls", 0) or 0) @@ -6164,6 +6336,38 @@ async def settle_approval_replay_calls(function_calls: Sequence[Content]) -> Non invocation_session=invocation_session, ) + async def settle_oauth_pause( + response: ChatResponse[Any], + result_messages: Sequence[Message], + *, + input_messages: Sequence[Message] | None = None, + ) -> ChatResponseUpdate | None: + nonlocal completed_usage + settlement = await self._settle_oauth_consent_results( + super_get_response=super_get_response, + response=response, + result_messages=result_messages, + options=options, + request_kwargs=request_kwargs, + compaction_strategy=compaction_strategy, + tokenizer=tokenizer, + invocation_session=invocation_session, + input_messages=input_messages, + ) + if settlement is None: + return None + if settlement.usage_details is not None: + completed_usage = add_usage_details(completed_usage, settlement.usage_details) + return ChatResponseUpdate( + response_id=settlement.response_id, + conversation_id=settlement.conversation_id, + contents=( + [Content.from_usage(copy.deepcopy(completed_usage), merge_mode="replace")] + if completed_usage is not None + else [] + ), + ) + # Apply limit decisions early to prevent execution during replay if limits are already breached. _apply_batch_limit_decision( "continue", @@ -6196,6 +6400,14 @@ async def settle_approval_replay_calls(function_calls: Sequence[Content]) -> Non total_function_calls, approval_processing.function_call_count, ) + if approval_processing.action == "return": + settlement_update = await settle_oauth_pause( + ChatResponse(messages=list(approval_processing.response_messages)), + approval_processing.response_messages, + input_messages=prepared_messages, + ) + if settlement_update is not None: + yield settlement_update for update in approval_processing.streaming_updates: yield update if approval_processing.action == "return": @@ -6352,6 +6564,10 @@ async def settle_approval_replay_calls(function_calls: Sequence[Content]) -> Non total_function_calls, function_processing.function_call_count, ) + if function_processing.action == "return": + settlement_update = await settle_oauth_pause(response, function_processing.response_messages) + if settlement_update is not None: + yield settlement_update for update in function_processing.streaming_updates: yield update if function_processing.action != "continue" and function_processing.action != "stop": diff --git a/python/packages/core/agent_framework/exceptions.py b/python/packages/core/agent_framework/exceptions.py index a908b5bce5d..93af9021388 100644 --- a/python/packages/core/agent_framework/exceptions.py +++ b/python/packages/core/agent_framework/exceptions.py @@ -194,6 +194,9 @@ class UserInputRequiredException(ToolException): invocation layer can propagate them instead of swallowing them as a generic tool error. ``Agent.as_tool()`` handles unresolved child function approvals separately and fails the delegated invocation. + An OAuth consent request also produces one terminal result for the interrupted + function call. The loop lets the authorized batch finish and preserves all + results before returning the consent requests. Args: contents: The user-input-request Content items from the sub-agent response. diff --git a/python/packages/core/agent_framework/foundry/__init__.py b/python/packages/core/agent_framework/foundry/__init__.py index 6664b6fb4ff..e34c1441fa8 100644 --- a/python/packages/core/agent_framework/foundry/__init__.py +++ b/python/packages/core/agent_framework/foundry/__init__.py @@ -10,6 +10,8 @@ - ``agent-framework-foundry-local`` """ +from __future__ import annotations + import importlib from typing import Any @@ -39,10 +41,7 @@ "CheckpointStoreProvider": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "FoundryCheckpointStore": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "ContextScopedStoreProvider": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), - "FoundryFunctionApprovalStore": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "FoundryRequestScope": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), - "FunctionApprovalStore": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), - "FunctionApprovalStoreProvider": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "FoundryToolbox": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "FoundryLocalChatOptions": ("agent_framework_foundry_local", "agent-framework-foundry-local"), "FoundryLocalClient": ("agent_framework_foundry_local", "agent-framework-foundry-local"), @@ -54,7 +53,10 @@ "RawFoundryChatClient": ("agent_framework_foundry", "agent-framework-foundry"), "RawFoundryEmbeddingClient": ("agent_framework_foundry", "agent-framework-foundry"), "InvocationsHostServer": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), + "InvocationsWorkflowHostServer": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "ResponsesHostServer": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), + "ResponsesRequest": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), + "ResponsesWorkflowHostServer": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "WorkflowSource": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "WorkflowTurn": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), "response_input_messages": ("agent_framework_foundry_hosting", "agent-framework-foundry-hosting"), diff --git a/python/packages/core/agent_framework/foundry/__init__.pyi b/python/packages/core/agent_framework/foundry/__init__.pyi index e5a86b75346..1cd4bff5293 100644 --- a/python/packages/core/agent_framework/foundry/__init__.pyi +++ b/python/packages/core/agent_framework/foundry/__init__.pyi @@ -37,11 +37,8 @@ from agent_framework_foundry_hosting import ( ContextScopedStoreProvider, FoundryAgentSessionStore, FoundryCheckpointStore, - FoundryFunctionApprovalStore, FoundryRequestScope, FoundryToolbox, - FunctionApprovalStore, - FunctionApprovalStoreProvider, InvocationsHostServer, ResponsesHostServer, StoreProvider, @@ -49,6 +46,15 @@ from agent_framework_foundry_hosting import ( WorkflowTurn, response_input_messages, ) +from agent_framework_foundry_hosting import ( + InvocationsWorkflowHostServer as InvocationsWorkflowHostServer, +) +from agent_framework_foundry_hosting import ( + ResponsesRequest as ResponsesRequest, +) +from agent_framework_foundry_hosting import ( + ResponsesWorkflowHostServer as ResponsesWorkflowHostServer, +) from agent_framework_foundry_local import ( FoundryLocalChatOptions, FoundryLocalClient, @@ -76,23 +82,23 @@ __all__ = [ "FoundryEmbeddingOptions", "FoundryEmbeddingSettings", "FoundryEvals", - "FoundryFunctionApprovalStore", "FoundryLocalChatOptions", "FoundryLocalClient", "FoundryLocalSettings", "FoundryMemoryProvider", "FoundryRequestScope", "FoundryToolbox", - "FunctionApprovalStore", - "FunctionApprovalStoreProvider", "GeneratedEvaluatorRef", "InvocationsHostServer", + "InvocationsWorkflowHostServer", "RawAnthropicFoundryClient", "RawFoundryAgent", "RawFoundryAgentChatClient", "RawFoundryChatClient", "RawFoundryEmbeddingClient", "ResponsesHostServer", + "ResponsesRequest", + "ResponsesWorkflowHostServer", "StoreProvider", "WorkflowSource", "WorkflowTurn", diff --git a/python/packages/core/tests/core/test_clients.py b/python/packages/core/tests/core/test_clients.py index bdb770dbc3f..d5663b8ba49 100644 --- a/python/packages/core/tests/core/test_clients.py +++ b/python/packages/core/tests/core/test_clients.py @@ -941,8 +941,13 @@ def terminal_tool() -> str: assert "London" in (response.messages[summary_index].text or "") assert sum(content.call_id == "call_1" for content in correlated_contents) == 2 assert sum(content.call_id == "call_2" for content in correlated_contents) == 2 - assert sum(content.call_id == "call_terminal" for content in correlated_contents) == ( - 1 if terminal_kind == "approval" else 2 + expected_terminal_types = { + "approval": ["function_call"], + "user-input": ["function_call", "function_result", "oauth_consent_request"], + "middleware-termination": ["function_call", "function_result"], + } + assert [content.type for content in correlated_contents if content.call_id == "call_terminal"] == ( + expected_terminal_types[terminal_kind] ) if terminal_kind == "approval": assert any( diff --git a/python/packages/core/tests/core/test_function_invocation_logic.py b/python/packages/core/tests/core/test_function_invocation_logic.py index 54fafe4c72b..9f052871383 100644 --- a/python/packages/core/tests/core/test_function_invocation_logic.py +++ b/python/packages/core/tests/core/test_function_invocation_logic.py @@ -1,6 +1,7 @@ # Copyright (c) Microsoft. All rights reserved. import asyncio +import copy import json import logging import math @@ -10814,7 +10815,8 @@ def guarded_input_tool() -> str: resumed_updates = [update async for update in resumed_stream] resumed_response = await resumed_stream.get_final_response() assert [[content.type for content in update.contents] for update in resumed_updates] == [ - ["oauth_consent_request", "oauth_consent_request"] + ["function_result"], + ["oauth_consent_request", "oauth_consent_request"], ] assert len(chat_client_base.streaming_responses) == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] else: @@ -10921,7 +10923,7 @@ def paused_tool() -> str: resumed_updates = [update async for update in resumed_stream] resumed_response = await resumed_stream.get_final_response() assert [(update.role, [content.type for content in update.contents]) for update in resumed_updates] == [ - ("tool", ["function_result"]), + ("tool", ["function_result", "function_result"]), ("assistant", ["oauth_consent_request"]), ] assert len(chat_client_base.streaming_responses) == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] @@ -10936,7 +10938,7 @@ def paused_tool() -> str: assert [ (message.role, [content.type for content in message.contents]) for message in resumed_response.messages ] == [ - ("tool", ["function_result"]), + ("tool", ["function_result", "function_result"]), ("assistant", ["oauth_consent_request"]), ] assert resumed_response.messages[0].contents[0].call_id == "call_completed" @@ -11494,6 +11496,597 @@ def test_dict_overwrites_existing_conversation_id(self): # endregion +@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"]) +@pytest.mark.parametrize("concurrent", [False, True], ids=["sequential", "concurrent"]) +@pytest.mark.parametrize("approval_resume", [False, True], ids=["ordinary", "approved-replay"]) +@pytest.mark.parametrize("later_outcome", ["result", "consent", "error"]) +async def test_oauth_consent_preserves_all_batch_results( + chat_client_base: SupportsChatGetResponse, + streaming: bool, + concurrent: bool, + approval_resume: bool, + later_outcome: str, +) -> None: + from agent_framework import function_middleware + from agent_framework._tools import _FUNCTION_INVOCATION_BUDGET_STATE_KEY + from agent_framework.exceptions import UserInputRequiredException + + executions: list[str] = [] + middleware_calls: list[str] = [] + + @function_middleware + async def record_functions(context: FunctionInvocationContext, call_next: Callable[[], Awaitable[None]]) -> None: + middleware_calls.append(context.function.name) + await call_next() + + @tool(approval_mode="always_require" if approval_resume else "never_require") + async def completed_tool() -> str: + executions.append("completed") + return "completed-result" + + @tool(approval_mode="always_require" if approval_resume else "never_require") + async def consent_tool() -> str: + executions.append("consent") + raise UserInputRequiredException([Content.from_oauth_consent_request("https://consent.example.com/authorize")]) + + @tool(approval_mode="always_require" if approval_resume else "never_require") + async def later_tool() -> str: + executions.append("later") + if later_outcome == "consent": + raise UserInputRequiredException([Content.from_oauth_consent_request("https://consent.example.com/later")]) + if later_outcome == "error": + raise RuntimeError("later tool failed") + return "later-result" + + function_calls = [ + Content.from_function_call(name, name, arguments={}, id=f"occurrence-{name}") + for name in ("completed_tool", "consent_tool", "later_tool") + ] + chat_client_base.function_invocation_configuration["allow_concurrent_invocation"] = concurrent # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + budget_state: dict[str, int] = {} + session = AgentSession() + options: ChatOptions = {"tools": [completed_tool, consent_tool, later_tool]} + client_kwargs: dict[str, Any] = { + "session": session, + "middleware": [record_functions], + _FUNCTION_INVOCATION_BUDGET_STATE_KEY: budget_state, + } + updates: list[ChatResponseUpdate] = [] + if streaming: + chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + [ChatResponseUpdate(role="assistant", contents=function_calls)] + ] + stream = chat_client_base.get_response( + [Message("user", contents=["run all"])], + stream=True, + options=options, + client_kwargs=client_kwargs, + ) + updates = [update async for update in stream] + response = await stream.get_final_response() + else: + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse(messages=[Message("assistant", contents=function_calls)]) + ] + response = await chat_client_base.get_response( + [Message("user", contents=["run all"])], + options=options, + client_kwargs=client_kwargs, + ) + + if approval_resume: + decisions = [ + content.to_function_approval_response(approved=True) + for message in response.messages + for content in message.contents + if content.type == "function_approval_request" + ] + assert len(decisions) == 3 + assert not executions + if streaming: + stream = chat_client_base.get_response( + [Message("user", contents=decisions)], + stream=True, + options=options, + client_kwargs=client_kwargs, + ) + updates = [update async for update in stream] + response = await stream.get_final_response() + else: + response = await chat_client_base.get_response( + [Message("user", contents=decisions)], + options=options, + client_kwargs=client_kwargs, + ) + if streaming: + streamed_results = [ + content for update in updates for content in update.contents if content.type == "function_result" + ] + assert [content.call_id for content in streamed_results] == [call.call_id for call in function_calls] + results = [ + content for message in response.messages for content in message.contents if content.type == "function_result" + ] + assert executions == ["completed", "consent", "later"] + assert middleware_calls == ["completed_tool", "consent_tool", "later_tool"] + assert [(result.call_id, result.id) for result in results] == [(call.call_id, call.id) for call in function_calls] + assert results[0].result == "completed-result" + assert "OAuth consent is required" in results[1].result + if later_outcome == "consent": + assert "OAuth consent is required" in results[2].result + elif later_outcome == "error": + assert results[2].exception is not None + else: + assert results[2].result == "later-result" + assert all("https://" not in result.result for result in results) + assert budget_state["total_function_calls"] == 3 + assert [ + content.consent_link + for message in response.messages + for content in message.contents + if content.type == "oauth_consent_request" + ] == ( + ["https://consent.example.com/authorize", "https://consent.example.com/later"] + if later_outcome == "consent" + else ["https://consent.example.com/authorize"] + ) + + +@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"]) +async def test_oauth_mixed_pause_retains_partial_input_and_budget( + chat_client_base: SupportsChatGetResponse, + streaming: bool, +) -> None: + from agent_framework.exceptions import UserInputRequiredException + + executions: list[str] = [] + requests: list[dict[str, Any]] = [] + + @chat_middleware + async def record_requests(context: ChatContext, call_next: Callable[[], Awaitable[None]]) -> None: + requests.append({"options": dict(context.options or {}), "messages": copy.deepcopy(context.messages)}) + await call_next() + + @tool + async def completed_tool() -> str: + executions.append("completed") + return "preserved-result" + + @tool + async def consent_tool() -> str: + executions.append("consent") + raise UserInputRequiredException([Content.from_oauth_consent_request("https://consent.example.com/authorize")]) + + @tool + async def input_tool() -> str: + executions.append("input") + raise UserInputRequiredException([Content("text", text="Supply input.", user_input_request=True)]) + + calls = [ + Content.from_function_call(call_id, name, arguments={}, id=f"occurrence-{call_id}") + for call_id, name in ( + ("completed", "completed_tool"), + ("consent", "consent_tool"), + ("input-1", "input_tool"), + ("input-2", "input_tool"), + ) + ] + chat_client_base.chat_middleware = [record_requests] # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + chat_client_base.function_invocation_configuration["max_function_calls"] = 4 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + if streaming: + chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + [ChatResponseUpdate(role="assistant", contents=calls, conversation_id="service-calls")], + ] + else: + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse(messages=[Message("assistant", contents=calls)], conversation_id="service-calls"), + ] + options: ChatOptions = {"tools": [completed_tool, consent_tool, input_tool]} + session = AgentSession() + if streaming: + first = await chat_client_base.get_response( + [Message("user", contents=["Run the tools."])], + stream=True, + options=options, + client_kwargs={"session": session}, + ).get_final_response() + else: + first = await chat_client_base.get_response( + [Message("user", contents=["Run the tools."])], options=options, client_kwargs={"session": session} + ) + assert any(content.type == "oauth_consent_request" for message in first.messages for content in message.contents) + assert len(requests) == 1 + options["conversation_id"] = "service-calls" + partial_input = Message( + "user", + contents=[ + Content.from_text("Keep the first instruction."), + Content.from_function_result("input-1", result="first-input"), + ], + ) + original_input = partial_input.to_dict() + if streaming: + await chat_client_base.get_response( + [partial_input], stream=True, options=options, client_kwargs={"session": session} + ).get_final_response() + else: + await chat_client_base.get_response([partial_input], options=options, client_kwargs={"session": session}) + assert len(requests) == 1 + assert partial_input.to_dict() == original_input + session = AgentSession.from_dict(session.to_dict()) + final_input = Message( + "user", + contents=[ + Content.from_text("Keep the last instruction."), + Content.from_function_result("input-2", result="last-input"), + ], + ) + if streaming: + final = await chat_client_base.get_response( + [final_input], stream=True, options=options, client_kwargs={"session": session} + ).get_final_response() + else: + final = await chat_client_base.get_response([final_input], options=options, client_kwargs={"session": session}) + assert not any(content.user_input_request for message in final.messages for content in message.contents) + assert len(requests) == 2 + assert requests[1]["options"]["tool_choice"] == "none" + model_messages = requests[1]["messages"] + results = [ + content for message in model_messages for content in message.contents if content.type == "function_result" + ] + assert [content.call_id for content in results] == [call.call_id for call in calls] + assert results[0].result == "preserved-result" + assert [content.result for content in results[2:]] == ["first-input", "last-input"] + assert all( + message.role == "tool" + for message in model_messages + if any(c.type == "function_result" for c in message.contents) + ) + assert "Keep the first instruction." in " ".join(message.text for message in model_messages) + assert "Keep the last instruction." in " ".join(message.text for message in model_messages) + assert not any( + content.type == "oauth_consent_request" for message in model_messages for content in message.contents + ) + assert executions == ["completed", "consent", "input", "input"] + + +@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"]) +@pytest.mark.parametrize("approval_resume", [False, True], ids=["ordinary", "approved-replay"]) +async def test_oauth_consent_reconciles_real_results_and_advances_service_history( + chat_client_base: SupportsChatGetResponse, + streaming: bool, + approval_resume: bool, +) -> None: + from agent_framework.exceptions import UserInputRequiredException + + requests: list[dict[str, Any]] = [] + + @chat_middleware + async def record_requests(context: ChatContext, call_next: Callable[[], Awaitable[None]]) -> None: + requests.append({ + "conversation_id": (context.options or {}).get("conversation_id"), + "tool_choice": (context.options or {}).get("tool_choice"), + "contents": [content for message in context.messages for content in message.contents], + }) + await call_next() + if len(requests) == 2: + context.result = settlement + + @tool(approval_mode="always_require" if approval_resume else "never_require") + async def completed_tool() -> str: + return "preserved-result" + + @tool(approval_mode="always_require" if approval_resume else "never_require") + async def consent_tool() -> str: + raise UserInputRequiredException([Content.from_oauth_consent_request("https://consent.example.com/authorize")]) + + function_calls = [ + Content.from_function_call("completed", "completed_tool", arguments={}), + Content.from_function_call("consent", "consent_tool", arguments={}), + ] + settlement = ChatResponse( + messages=[Message("assistant", contents=["hidden settlement reply"])], + response_id="resp_settled", + conversation_id="resp_settled", + usage_details={"input_token_count": 3, "output_token_count": 2}, + ) + chat_client_base.chat_middleware = [record_requests] # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + if streaming: + chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + [ + ChatResponseUpdate( + role="assistant", + contents=function_calls, + response_id="resp_calls", + conversation_id="resp_calls", + ) + ] + ] + chat_client_base.run_responses = [settlement] # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + else: + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse( + messages=[Message("assistant", contents=function_calls)], + response_id="resp_calls", + conversation_id="resp_calls", + ), + settlement, + ] + session = AgentSession() + options: ChatOptions = {"tools": [completed_tool, consent_tool]} + updates: list[ChatResponseUpdate] = [] + if streaming: + stream = chat_client_base.get_response( + [Message("user", contents=["run both"])], + stream=True, + options=options, + client_kwargs={"session": session}, + ) + updates = [update async for update in stream] + response = await stream.get_final_response() + else: + response = await chat_client_base.get_response( + [Message("user", contents=["run both"])], options=options, client_kwargs={"session": session} + ) + + if approval_resume: + decisions = [ + content.to_function_approval_response(approved=True) + for message in response.messages + for content in message.contents + if content.type == "function_approval_request" + ] + assert len(decisions) == 2 + assert len(requests) == 1 + assert session.service_session_id == "resp_calls" + options["conversation_id"] = "resp_calls" + resume_message = Message("user", contents=[Content.from_text("Keep this instruction."), *decisions]) + if streaming: + stream = chat_client_base.get_response( + [resume_message], + stream=True, + options=options, + client_kwargs={"session": session}, + ) + updates = [update async for update in stream] + response = await stream.get_final_response() + else: + response = await chat_client_base.get_response( + [resume_message], + options=options, + client_kwargs={"session": session}, + ) + if streaming: + assert any(content.type == "oauth_consent_request" for update in updates for content in update.contents) + assert len(requests) == 2 + assert requests[1]["conversation_id"] == "resp_calls" + assert requests[1]["tool_choice"] == "none" + settled_results = [content for content in requests[1]["contents"] if content.type == "function_result"] + assert settled_results[0].result == "preserved-result" + assert [content.call_id for content in settled_results] == ["completed", "consent"] + assert not any(content.type == "oauth_consent_request" for content in requests[1]["contents"]) + if approval_resume: + assert any(content.text == "Keep this instruction." for content in requests[1]["contents"]) + else: + assert [content.type for content in requests[1]["contents"]] == ["function_result", "function_result"] + assert response.conversation_id == session.service_session_id == "resp_settled" + assert response.response_id == "resp_settled" + assert "hidden settlement reply" not in response.text + + +@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"]) +@pytest.mark.parametrize("outcome", ["failure", "cancellation", "unexpected-call", "unfinished"]) +async def test_oauth_consent_settlement_failure_withholds_user_input( + chat_client_base: SupportsChatGetResponse, + streaming: bool, + outcome: str, +) -> None: + from agent_framework.exceptions import UserInputRequiredException + + requests = 0 + executions = 0 + + @chat_middleware + async def settlement_outcome(context: ChatContext, call_next: Callable[[], Awaitable[None]]) -> None: + nonlocal requests + requests += 1 + if requests == 2: + if outcome == "failure": + raise RuntimeError("settlement refused") + if outcome == "cancellation": + raise asyncio.CancelledError + context.result = ChatResponse( + messages=( + [ + Message( + "assistant", contents=[Content.from_function_call("new-call", "consent_tool", arguments={})] + ) + ] + if outcome == "unexpected-call" + else [] + ), + conversation_id="resp_accepted", + continuation_token={} if outcome == "unfinished" else None, + ) + return + await call_next() + + @tool(approval_mode="never_require") + async def consent_tool() -> str: + nonlocal executions + executions += 1 + raise UserInputRequiredException([Content.from_oauth_consent_request("https://consent.example.com/authorize")]) + + function_call = Content.from_function_call("consent", "consent_tool", arguments={}) + chat_client_base.chat_middleware = [settlement_outcome] # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + if streaming: + chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + [ChatResponseUpdate(role="assistant", contents=[function_call], conversation_id="resp_calls")] + ] + else: + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse(messages=[Message("assistant", contents=[function_call])], conversation_id="resp_calls") + ] + session = AgentSession() + updates: list[ChatResponseUpdate] = [] + expected_error = asyncio.CancelledError if outcome == "cancellation" else RuntimeError + with pytest.raises(expected_error): + if streaming: + async for update in chat_client_base.get_response( + [Message("user", contents=["run"])], + stream=True, + options={"tools": [consent_tool]}, + client_kwargs={"session": session}, + ): + updates.append(update) + else: + await chat_client_base.get_response( + [Message("user", contents=["run"])], + options={"tools": [consent_tool]}, + client_kwargs={"session": session}, + ) + + assert requests == 2 + assert executions == 1 + assert not any(content.type == "oauth_consent_request" for update in updates for content in update.contents) + assert session.service_session_id == ( + "resp_accepted" if outcome in ("unexpected-call", "unfinished") else "resp_calls" + ) + + +@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"]) +async def test_oauth_consent_preserves_already_dispatched_sibling( + chat_client_base: SupportsChatGetResponse, streaming: bool +) -> None: + from agent_framework.exceptions import UserInputRequiredException + + started = asyncio.Event() + finish = asyncio.Event() + completed = False + + @tool(approval_mode="never_require") + async def inflight_tool() -> str: + nonlocal completed + started.set() + await finish.wait() + completed = True + return "settled sibling" + + @tool(approval_mode="never_require") + async def consent_tool() -> str: + await started.wait() + finish.set() + raise UserInputRequiredException([Content.from_oauth_consent_request("https://consent.example.com/authorize")]) + + function_calls = [ + Content.from_function_call("inflight", "inflight_tool", arguments={}), + Content.from_function_call("consent", "consent_tool", arguments={}), + ] + options: ChatOptions = {"tools": [inflight_tool, consent_tool]} + if streaming: + chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + [ChatResponseUpdate(role="assistant", contents=function_calls)] + ] + stream = chat_client_base.get_response([Message("user", contents=["run"])], stream=True, options=options) + _ = [update async for update in stream] + response = await stream.get_final_response() + else: + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse(messages=[Message("assistant", contents=function_calls)]) + ] + response = await chat_client_base.get_response([Message("user", contents=["run"])], options=options) + + results = [ + content for message in response.messages for content in message.contents if content.type == "function_result" + ] + assert completed + assert [content.call_id for content in results] == ["inflight", "consent"] + assert results[0].result == "settled sibling" + + +@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"]) +async def test_provider_oauth_consent_is_not_locally_settled( + chat_client_base: SupportsChatGetResponse, streaming: bool +) -> None: + @tool(approval_mode="never_require") + async def unused_tool() -> str: + pytest.fail("A provider-owned consent request must not execute a local tool.") + + contents = [Content.from_oauth_consent_request("https://consent.example.com/provider")] + session = AgentSession() + if streaming: + chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + [ChatResponseUpdate(role="assistant", contents=contents, conversation_id="resp_provider_consent")] + ] + stream = chat_client_base.get_response( + [Message("user", contents=["run"])], + stream=True, + options={"tools": [unused_tool]}, + client_kwargs={"session": session}, + ) + _ = [update async for update in stream] + response = await stream.get_final_response() + else: + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse(messages=[Message("assistant", contents=contents)], conversation_id="resp_provider_consent") + ] + response = await chat_client_base.get_response( + [Message("user", contents=["run"])], + options={"tools": [unused_tool]}, + client_kwargs={"session": session}, + ) + assert chat_client_base.call_count == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + assert session.service_session_id == "resp_provider_consent" + assert [content.type for message in response.messages for content in message.contents] == ["oauth_consent_request"] + + +@pytest.mark.parametrize("duplicate_result", [False, True]) +async def test_oauth_consent_supplied_terminal_result_stays_single_and_immutable( + chat_client_base: SupportsChatGetResponse, duplicate_result: bool +) -> None: + from agent_framework.exceptions import UserInputRequiredException + + contents = [Content.from_function_result("unbound", result="custom consent result", id="caller-result")] + if duplicate_result: + contents.append(Content.from_function_result("other", result="duplicate")) + contents.append(Content.from_oauth_consent_request("https://consent.example.com/authorize")) + original = [content.to_dict() for content in contents] + + @tool(approval_mode="never_require") + async def request_consent() -> str: + raise UserInputRequiredException(contents) + + chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] + ChatResponse( + messages=[ + Message( + "assistant", + contents=[Content.from_function_call("bound", "request_consent", arguments={}, id="occurrence")], + ) + ] + ) + ] + if duplicate_result: + with pytest.raises(ValueError, match="at most one terminal function result") as failure: + await chat_client_base.get_response( + [Message("user", contents=["run"])], options={"tools": [request_consent]} + ) + assert isinstance(failure.value.__cause__, UserInputRequiredException) + else: + response = await chat_client_base.get_response( + [Message("user", contents=["run"])], options={"tools": [request_consent]} + ) + results = [ + content + for message in response.messages + for content in message.contents + if content.type == "function_result" + ] + assert [(result.call_id, result.id, result.result) for result in results] == [ + ("bound", "occurrence", "custom consent result") + ] + assert [content.to_dict() for content in contents] == original + + async def test_user_input_request_propagates_through_as_tool(chat_client_base: SupportsChatGetResponse): """Test that user_input_request content from a sub-agent wrapped as a tool propagates to the parent response.""" from agent_framework.exceptions import UserInputRequiredException diff --git a/python/packages/core/tests/core/test_harness_tool_approval.py b/python/packages/core/tests/core/test_harness_tool_approval.py index d2170fbff4d..329aaa83ba6 100644 --- a/python/packages/core/tests/core/test_harness_tool_approval.py +++ b/python/packages/core/tests/core/test_harness_tool_approval.py @@ -1568,7 +1568,13 @@ def oauth_tool() -> str: "https://example.com/consent-2", "https://example.com/consent-3", ] - assert resumed_response.messages[0].role == "assistant" + assert [ + (message.role, [content.type for content in message.contents]) for message in resumed_response.messages + ] == [ + ("tool", ["function_result"]), + ("assistant", ["oauth_consent_request", "oauth_consent_request", "oauth_consent_request"]), + ] + assert resumed_response.messages[0].contents[0].call_id == "call_oauth" assert calls == 1 diff --git a/python/packages/foundry_hosting/README.md b/python/packages/foundry_hosting/README.md index 068188eb61f..f5fe6c49b17 100644 --- a/python/packages/foundry_hosting/README.md +++ b/python/packages/foundry_hosting/README.md @@ -2,6 +2,123 @@ This package provides the integration of Agent Framework agents and workflows with the Foundry Agent Server, which can be hosted on Foundry infrastructure. +Implementation is grouped by protocol: `_responses_agent.py` and `_responses_workflow.py` define the Responses hosts, +and `_responses_shared.py` contains their wire and streaming helpers. Invocations uses the same `_invocations_agent.py`, +`_invocations_workflow.py`, and `_invocations_shared.py` layout. Cross-protocol request contracts, option validation, +and agent/workflow factories live in `_shared.py`. Applications use the public package imports below. +Both Invocations hosts inherit directly from the SDK host; shared functions handle identity, wire conversion, +and streaming cleanup without a combined agent/workflow host. + +## Choose the host + +Use `ResponsesHostServer` or `InvocationsHostServer` for agents. Use +`ResponsesWorkflowHostServer` or `InvocationsWorkflowHostServer` for native workflows. +Each workflow host requires a request-aware sync/async factory and an explicit parser. +All four hosts remain in `agent-framework-foundry-hosting`, with matching lazy exports +under `agent_framework.foundry`. Invocations and Toolbox retain their experimental boundary; +the split does not promote them. + +**Breaking migration:** replace `ResponsesHostServer(workflow=..., parse_response=...)` with +`ResponsesWorkflowHostServer(workflow=..., parse_input=...)`, and replace +`InvocationsHostServer(workflow=..., parse_request=...)` with +`InvocationsWorkflowHostServer(workflow=..., parse_input=...)`. +There is no combined-host compatibility shim. Agent host construction still uses `agent=`. +All custom parsers now use `parse_input`, including agent parsers on `InvocationsHostServer`. +Rename `HostedResponseRequest` imports and annotations to `ResponsesRequest`, including factories and option hooks. +`InvocationRun` and `WorkflowTurn` are unchanged. Workflow parsers still adapt either typed start input +or a complete batch of pending replies; they are not limited to initial input or raw JSON deserialization. +The old parser keywords and request-type name have no compatibility aliases. + +Agent history/background source literals also change without aliases: + +| Setting | Previous literal | New literal | +| --- | --- | --- | +| `history_source` | `"agent_server"` | `"host_managed"` | +| `history_source` | `"agent"` | `"agent_managed"` | +| `history_source` | `"service"` | `"inference_service"` | +| `background_source` | `"agent_server"` | `"host_managed"` | +| `background_source` | `"provider"` | `"inference_service"` | + +These are terminology changes only. The default remains host-managed history and background execution. +`agent_managed` retains developer-owned provider **or** service storage, not just `HistoryProvider` storage. +`response_store=` and the wire-level `store` flag keep their existing meanings. +The deprecated constructor `store=` backend alias is removed. Replace it with `response_store=`; +even `store=None` is rejected. This does not rename the caller's per-request `store` flag. + +`WorkflowSource` now describes a factory only; passing a built workflow fails at construction, +including local one-shot use. Move graph and executor construction inside the factory; +returning a new wrapper around shared executors does not isolate execution state. + +Workflow hosts expose checkpoint storage, not agent-session storage or agent history/background +policies. Responses workflows retain `prepare_options`; Invocations workflows use the parser's +explicit workflow kwargs instead. Regular-agent approval state belongs to the native +agent and its `AgentSession`; no separate approval-store API is exposed. +Agent hosts reject `WorkflowAgent`, whether supplied directly or returned by a factory. +`checkpoint_store_provider` is available only on workflow hosts. +No stored state is converted between native workflow and agent execution models. + +## Explicit host configuration + +Responses hosts construct the upstream +[`ResponsesServerOptions`](https://pypi.org/project/azure-ai-agentserver-responses/) internally. +Use first-class host parameters rather than importing an SDK options bag: + +| Parameter | Available on | Meaning and SDK consumer | +| --- | --- | --- | +| `log_level=None` | All four hosts | A literal `"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`, or `"CRITICAL"`, forwarded unchanged to the AgentServer SDK constructor. `None` preserves the SDK default. | +| `max_history_items: int \| None = None` | Responses agent host, with `history_source="host_managed"` | A positive cap on prior outer history items, passed to the SDK history loader as `default_fetch_history_count`. `None` supplies no override and inherits the installed supported SDK's default. | +| `sse_keep_alive_interval_seconds: int \| None = None` | All four hosts with framed SSE | A positive streaming heartbeat interval in seconds. Responses forwards it to SDK streaming routes; Invocations sets the host's SDK heartbeat configuration. `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. | +| `shutdown_grace_period_seconds: int = 10` | Both Responses hosts | A positive shutdown wait in seconds. The SDK also uses it when DELETE waits for response persistence. It is not a request or inference timeout. | +| `resilient_background: bool = False` | Both Responses hosts | Uses the SDK's recovery setting without renaming it. It permits eligible recovery, not background execution or exactly-once side effects. | + +`max_history_items=None` does **not** promise unlimited history or map to SDK `-1`. +Supported SDK versions can have different defaults. Explicit caps count prior history items, not tokens, +turns, retention, or workflow checkpoints. Other agent history modes and workflow hosts do not accept a cap. + +Explicit heartbeat intervals must be positive: zero, negative values, and booleans are invalid, not +force-disable switches or requests to inherit the environment. The SDK still parses its environment +configuration during construction; a malformed `SSE_KEEPALIVE_INTERVAL` can fail even with an explicit interval. +The agent-only `legacy_wire_format=True` disables heartbeats and cannot be combined with an explicit interval. + +For regular-agent recovery, select both `history_source="inference_service"` and +`background_source="inference_service"`. Native workflows recover from their own checkpoints. +The SDK recovery setting itself is provider-independent; the inference-service requirement is a limitation +of the regular-agent adapter, which currently resumes only saved provider continuations. +Follow the SDK's task-subsystem and persistent-store requirements. Neither outer workflow background execution nor +recovery permission enables provider background on constituent agents. + +Host `default_model`, `create_span_hook`, `additional_server_version`, and `steerable_conversations` +are not first-class parameters. Framework-generated SDK options leave model defaults, span callbacks, and +custom version stamping unset, and keep steering disabled. Framework observability is unchanged. +Configure inference defaults on the agent or client. The former SDK `default_model` could fill the outer +request model and thereby override inference; it did not select a hosted agent. Removing that host fallback +does not change caller-supplied request models, option-hook timing, option precedence, or routing. + +### SDK constructor arguments + +All four hosts accept `agent_server_kwargs: Mapping[str, Any] | None = None`. +This caller-owned mapping is expanded only into the corresponding AgentServer SDK constructor. +Hosting does not inspect, validate, filter, or mutate its contents. Python and the SDK report invalid +arguments, including collisions with arguments the framework already supplies; there is no custom merge policy. +An SDK argument reaching this boundary does not mean every adapter path supports its behavior. + +Move SDK-specific top-level arguments into this mapping. For an application-owned `agent` and `asyncapi_spec`: + +```python +from agent_framework_foundry_hosting import InvocationsHostServer + +# Before: InvocationsHostServer(agent=agent, asyncapi_spec_json=asyncapi_spec) +server = InvocationsHostServer( + agent=agent, + agent_server_kwargs={"asyncapi_spec_json": asyncapi_spec}, +) +``` + +The constructors no longer accept arbitrary `**kwargs`. Removed top-level names, including `options=` +and old parser/storage aliases, raise Python `TypeError` before construction, even when their value is `None`. +Use the first-class parameters above for framework-supported configuration. Runtime model options, +`client_kwargs`, and `function_invocation_kwargs` keep their existing meanings and validation. + ## Toolbox tool approvals `FoundryToolbox` uses each discovered tool's `_meta.tool_configuration.require_approval` when @@ -27,53 +144,61 @@ general-purpose MCP approval policy. ## Agent instances and factories -`ResponsesHostServer` and `InvocationsHostServer` accept an agent instance or a zero-argument callable through the -existing `agent` parameter. The callable may be synchronous or asynchronous, must return an object implementing -`SupportsAgentRun`, and is invoked once for each request: +`ResponsesHostServer` and `InvocationsHostServer` accept an agent instance or a callable through the +`agent` parameter. The callable may be synchronous or asynchronous, must return an object implementing +`SupportsAgentRun`, and is invoked once for each request as `factory(request=request)`. +Factories must declare a required parameter named `request`, either keyword-only or positional-or-keyword. +Additional parameters may have defaults, and the factory may ignore the request. +Zero-argument factories and factories with only optional configuration parameters are rejected at construction; +add a required `request` parameter when migrating them. Responses factories receive a +`ResponsesRequest`, with trusted scope, current-turn input helpers, and translated caller options. +They run before `prepare_options`; the hook can subsequently replace those options. +Invocations factories receive the Starlette `Request` after parsing and option preparation. +Factory exceptions are not retried with another call shape. ```python server = ResponsesHostServer(agent=create_agent) ``` +For request-aware construction: + +```python +from agent_framework_foundry_hosting import ResponsesRequest, ResponsesHostServer + + +async def create_agent(*, request: ResponsesRequest): + return await build_agent_for_user(request.scope.user_id) + + +server = ResponsesHostServer(agent=create_agent) +``` + Passing an instance reuses that object for the lifetime of the host. Pass a callable when the agent keeps mutable state -outside `AgentSession`. The Responses host continues regular agents through its existing session store and workflow -agents through its existing checkpoint store. A callable does not make arbitrary instance fields persistent; state +outside `AgentSession`. Agent hosts continue agents through their session stores. A callable does not make arbitrary +instance fields persistent; state needed by later requests must remain in the supported stores. -### Hosting a `WorkflowAgent` is deprecated +### Migrate `WorkflowAgent` hosting -Hosting a `WorkflowAgent` (for example `workflow.as_agent()`) through `agent=` is deprecated and should be avoided, and -both `ResponsesHostServer` and `InvocationsHostServer` emit a `DeprecationWarning` for it and log the same message at -`WARNING` level, once per host. A `WorkflowAgent` is stateful: its workflow state stays in memory between runs, so one -instance must never serve requests from different users or conversations. Host the workflow natively with `workflow=` -and a request-aware factory instead; see +`ResponsesHostServer` and `InvocationsHostServer` reject `WorkflowAgent` instances and factory results. +Host the workflow natively with a request-aware factory returning a built `Workflow`, not `workflow.as_agent()`. +Workflow hosts also reject factories returning agents. See [Native Responses workflows](#native-responses-workflows) and [Native Invocations workflows](#native-invocations-workflows): ```python # build_workflow(request) returns a freshly built Workflow for each request. -server = ResponsesHostServer(workflow=build_workflow, parse_response=parse_response) +server = ResponsesWorkflowHostServer(workflow=build_workflow, parse_input=parse_input) ``` -Until you migrate `ResponsesHostServer`, pass a callable that builds a new workflow, executors, and wrapped agents for -every request, and never return a shared instance. Keep the workflow name and executor IDs stable so later requests can -find and restore its checkpoints: - -```python -def create_agent(): - return build_workflow().as_agent() - - -server = ResponsesHostServer(agent=create_agent) -``` - -`InvocationsHostServer` does not restore workflow checkpoints for agents, so a per-request factory only suits stateless, -single-turn workflows there. Use `workflow=` for any workflow that pauses or spans turns. +Build fresh workflows, executors, and agents for every request. Keep the workflow name and executor IDs stable so +later requests can find and restore checkpoints. Existing wrapper history, context providers, and event semantics +are not automatically unwrapped or migrated. ## Native Responses workflows -Use `workflow=` to host a built `Workflow` directly, without `.as_agent()`. -`parse_response` is required and returns exactly one typed start input or a +Use `ResponsesWorkflowHostServer` to host a native `Workflow` through a request-aware factory, without `.as_agent()`. +`parse_input` is required and returns exactly one typed start input or a complete batch of validated pending replies: ```python @@ -81,8 +206,8 @@ from pydantic import BaseModel from agent_framework_foundry_hosting import ( CheckpointStoreProvider, - HostedResponseRequest, - ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, WorkflowTurn, ) @@ -91,33 +216,44 @@ class Ticket(BaseModel): text: str -def build_workflow(request: HostedResponseRequest): +def build_workflow(request: ResponsesRequest): return build_fresh_graph() # stable workflow/executor IDs; fresh mutable resources -async def parse_response(request: HostedResponseRequest) -> WorkflowTurn[Ticket]: +async def parse_input(request: ResponsesRequest) -> WorkflowTurn[Ticket]: items = await request.get_input_items() if any(item.get("type") in ("function_call_output", "mcp_approval_response") for item in items): return WorkflowTurn(responses=await request.get_workflow_responses()) return WorkflowTurn(input=Ticket.model_validate_json(await request.get_input_text() or "")) -ResponsesHostServer( +ResponsesWorkflowHostServer( workflow=build_workflow, - parse_response=parse_response, + parse_input=parse_input, checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{Ticket.__module__}:{Ticket.__qualname__}"], ), ) ``` -A direct built workflow is single-use. Use a request-aware sync or async -factory for continuation, approval/user-input pauses, or background recovery. +Every native workflow host requires a request-aware sync or async factory, +including local one-shot execution, continuation, approval/user-input pauses, and background recovery. The factory must return a freshly built graph and freshly owned executors, agents, clients, context providers, and mutable tools; hosting rejects known sharing rather than cloning or unwrapping it. Workflow names and executor IDs must remain stable so the exact scoped checkpoint can be restored. +Construct tool objects inside the workflow factory as the default. A module-level +`@tool` decorator constructs one shared `FunctionTool`, including its invocation +counters and configured lifetime limits. The current ownership guard rejects +reusing that object across native workflow requests, even if its function is pure. + +A plain Python function can be defined outside the factory and wrapped with +`tool(function)` inside it to create a fresh tool object for each workflow. +This does not copy state captured by the function, bound objects, wrapped agents, +or connections. Developers must manage that state and its isolation explicitly; +moving only the wrapper construction does not make shared mutable resources safe. + Native workflow state is scoped to the trusted platform user plus Foundry sandbox. Each outer stored `response.id` is bound to its exact MAF checkpoint, graph identity, lineage, pending reply authority, output, and usage. The host @@ -136,15 +272,13 @@ provider continuation is rejected before output is paired or committed. For legacy message-input workflows, `response_input_messages(request)` converts only the current Responses turn to `list[Message]`. It does not load outer -history or decode pending replies. Existing `agent=workflow.as_agent()` hosting -remains for this beta with a once-per-host deprecation warning because wrapper -context providers, history, event projection, and request-info translation are -real semantics and are not silently unwrapped. See -[Hosting a `WorkflowAgent` is deprecated](#hosting-a-workflowagent-is-deprecated) -for the instance-sharing restriction. - -With `resilient_background=True`, the application must also enable the -AgentServer resilient task subsystem. Host-owned checkpoint/output pairs +history or decode pending replies. `agent=workflow.as_agent()` is no longer supported. +Wrapper context providers, history, event projection, and request-info translation are +not silently unwrapped. See [Migrate `WorkflowAgent` hosting](#migrate-workflowagent-hosting). + +With `resilient_background=True`, native workflows recover through the +AgentServer resilient task subsystem; SDK-specific task initialization and persistence requirements still apply. +Host-owned checkpoint/output pairs recover emitted output and usage without selecting newer unpaired workflow state. This is not an exactly-once guarantee for external tools: make side-effecting operations idempotent. Legacy unscoped workflow state is not @@ -175,21 +309,21 @@ MAF session and approval state is saved. The host's `history_source` independent | `history_source` | What the model receives | Inner storage on caller `store=True` | | --- | --- | --- | -| `"agent_server"` (default) | Prior outer Responses transcript plus new input | Disabled. Storing clients run with `store=False`; non-storing clients receive no storage option. | -| `"service"` | New input only | Enabled. The service-issued `AgentSession.service_session_id` is saved privately under the outer response ID or conversation. | -| `"agent"` | New input only; the agent chooses how to load history | Developer-owned: `HistoryProvider` with `default_options={"store": False}` loads from host-persisted `AgentSession.state`, **or** a storing client with `default_options={"store": True}` uses downstream service history. The existing behavior is unchanged. | +| `"host_managed"` (default) | Prior outer Responses transcript plus new input | Disabled. Storing clients run with `store=False`; non-storing clients receive no storage option. | +| `"inference_service"` | New input only | Enabled. The service-issued `AgentSession.service_session_id` is saved privately under the outer response ID or conversation. | +| `"agent_managed"` | New input only; the agent chooses how to load history | Developer-owned: `HistoryProvider` with `default_options={"store": False}` loads from host-persisted `AgentSession.state`, **or** a storing client with `default_options={"store": True}` uses downstream service history. The existing behavior is unchanged. | For example, suppose the first stored response answers **"My name is Ada"**, then the caller sends **"What is my name?"** with `previous_response_id` set to that response's **outer** `response.id`: -- **`"agent_server"`:** The model receives the first user input, the first assistant output, and the new +- **`"host_managed"`:** The model receives the first user input, the first assistant output, and the new question. The host reconstructs that transcript from the Responses store; the inner service does not retain it. -- **`"service"`:** The model receives only the new question as *request input*, along with the +- **`"inference_service"`:** The model receives only the new question as *request input*, along with the private `AgentSession.service_session_id` from the first turn. The downstream service retrieves its own transcript. A second branch from the first response cannot safely reuse that service thread and is rejected. -- **`"agent"`:** The model receives the new question. With `InMemoryHistoryProvider` and agent +- **`"agent_managed"`:** The model receives the new question. With `InMemoryHistoryProvider` and agent default `store=False`, the provider adds earlier messages from the saved MAF session; with agent default `store=True`, the downstream service owns the prior transcript instead. @@ -202,39 +336,39 @@ Choose **one** mode when constructing each host; do not reuse the same `Agent` i For example, to use downstream service history: ```python -server = ResponsesHostServer(agent=agent, history_source="service") +server = ResponsesHostServer(agent=agent, history_source="inference_service") ``` -Omitting `history_source` selects `"agent_server"`. To use a provider in `"agent"` mode, +Omitting `history_source` selects `"host_managed"`. To use a provider in `"agent_managed"` mode, configure that agent with `store=False` as shown in [agent_history.py](../../samples/04-hosting/foundry-hosted-agents/responses/basic/agent_history.py). -`"agent_server"` and `"service"` reject a load-enabled `HistoryProvider` alongside their own history source; -`"agent_server"` also rejects default downstream continuation IDs. Those modes require a `RawAgent` with a client declaring -`STORES_BY_DEFAULT`; `"service"` requires a storing client that returns a private continuation ID. Hosting may +`"host_managed"` and `"inference_service"` reject a load-enabled `HistoryProvider` alongside their own history source; +`"host_managed"` also rejects default downstream continuation IDs. Those modes require a `RawAgent` with a client declaring +`STORES_BY_DEFAULT`; `"inference_service"` requires a storing client that returns a private continuation ID. Hosting may add a transient in-memory provider to support function-call loops, but **never edits `agent.default_options`**. The host owns the provided agent instance and any providers it adds; do not reuse it with another host. A factory creates an independent agent for each request. -The existing `history_source="agent"` still preserves the agent's own provider **or** service storage defaults +The existing `history_source="agent_managed"` still preserves the agent's own provider **or** service storage defaults on stored requests, including `default_options={"store": True}`. Custom `SupportsAgentRun` implementations can continue using that mode for stored requests; it is not a forced-provider mode. -The outer storage-backend constructor argument is now `response_store=`. The old `store=` backend -argument remains an alias with its own once-per-host deprecation warning; supplying both is an error. -Neither constructor argument sets the caller's per-request `store` flag. +The outer storage-backend constructor argument is `response_store=`. The old `store=` backend +argument is rejected, including `None` and calls that also supply `response_store=`. +The backend selector does not set the caller's per-request `store` flag. `store=False` returns a one-shot response without **writing** host-managed session, conversation, or approval state; it also disables downstream service storage regardless of the developer's defaults. Unsafe custom agents, external history providers that store messages, and fixed downstream continuation defaults fail with an actionable error instead of silently persisting. An unstored service-mode request cannot resume a private service thread. -`history_source="agent"` also rejects an unstored continuation if its restored session uses downstream storage. Application-owned +`history_source="agent_managed"` also rejects an unstored continuation if its restored session uses downstream storage. Application-owned tools and external services may still have their own side effects. `background=True` requires outer `store=True`. Outer background work always uses the caller-visible `response.id` for polling; it does not enable provider-native -background automatically. `background_source="agent_server"` (default) uses only the outer background worker. -`background_source="provider"` is a separate opt-in for `history_source="service"` with a storing +background automatically. `background_source="host_managed"` (default) uses only the outer background worker. +`background_source="inference_service"` is a separate opt-in for `history_source="inference_service"` with a storing Responses client. Its private continuation token is saved under the outer ID and never returned to the caller. -Use `ResponsesServerOptions(resilient_background=True)` to permit recovery from a **saved** token; a crash before +Use `resilient_background=True` to permit recovery from a **saved** token; a crash before the token is saved cannot safely restart the inner job. Completed polling output, including local function calls, results, and usage, is saved together with the next token in the private response-ID snapshot before emission. Outer output checkpoints record which saved batches have been emitted, so recovery restores their usage and @@ -247,18 +381,19 @@ another background response and saves its next token. A crash after a local tool next token is saved can still repeat the tool on recovery; use idempotent tools or avoid provider background for side-effecting local tools. This mechanism does not provide exactly-once tool execution. Provider background and steering cannot be combined. -Regular agent runs without this opt-in are not crash-replayable. **Steering is temporarily unavailable:** -`steerable_conversations=True` fails during host construction, before enabling the process-wide TaskManager. +Regular agent runs without this opt-in are not crash-replayable. **Steering is unavailable:** +`steerable_conversations` and the upstream `options` bag are not host parameters; framework-generated SDK +options keep steering disabled. The current AgentServer SDK retains unbounded futures for rejected turns when its steering queue fills. Do not -use a queue-length precheck: another worker can append before it. The guard can be removed only after -[Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233) ships in an -official `azure-ai-agentserver-core` wheel, the minimum dependency and `uv.lock` are updated, and a -concurrent queue-overflow regression proves rejected turns leave no pending futures. No future SDK -version is assumed. Non-steerable background polling and legacy `WorkflowAgent` dispatch are unchanged. +use a queue-length precheck: another worker can append before it. Future steering support requires an +official `azure-ai-agentserver-core` wheel containing +[Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233), updated minimum +dependencies and `uv.lock`, and a concurrent queue-overflow regression proving rejected turns leave no pending futures. No future SDK +version is assumed. Non-steerable agent background polling is unchanged. Native CreateResponse generation fields become MAF runtime options (notably `max_output_tokens` -> `max_tokens` and `parallel_tool_calls` -> `allow_multiple_tool_calls`). Flattened OpenAI `extra_body` fields overlay translated keys -**last**. A sync or async `prepare_options(request: HostedResponseRequest, options: dict)` hook can remove or replace +**last**. A sync or async `prepare_options(request: ResponsesRequest, options: dict)` hook can remove or replace *caller* options before `Agent.run`; removed values fall back to the developer's unchanged agent defaults. Hosting filters caller platform IDs and private continuation/storage controls from model options and rejects attempts to reintroduce them through the hook. Nested `extra_body` transport overrides are rejected for both caller @@ -360,7 +495,7 @@ conversation or response ID. | MAF `AgentSession.session_id` / `service_session_id` | Inner agent state and optional downstream service continuation; neither identifies the Foundry sandbox. | | Workflow checkpoint ID | Inner workflow state within a checkpoint context, not a Foundry session ID. | -The default hosted MAF session, checkpoint, and approval stores use a `v2` namespace +The default hosted MAF session and checkpoint stores use a `v2` namespace derived from the hashed platform identity, with `user_isolation=True` and an explicit platform `call_id` on each item operation. Checkpoint context IDs are hashed as well. The same user in two hosted sandboxes cannot read the other's default MAF state. @@ -390,7 +525,7 @@ Invocations sessions use the separate `invocation_sessions` store. Each stored agent turn saves a snapshot under its **own** outer `response.id`. For a named `conversation`, a separate mutable conversation-head key is also updated. Loaded MAF sessions use PR1's ETag condition for that key: a competing turn that advanced the head causes a visible conflict rather than a stale overwrite. A superseded -steered turn saves its response snapshot but skips the head update. In `"service"` or `"agent"` history mode, a +steered turn saves its response snapshot but skips the head update. In `"inference_service"` or `"agent_managed"` history mode, a stored named-conversation turn first claims that head with a conditional write *before* calling the inner agent. Another request that read the old head loses the CAS; one that reads the claim fails before touching the provider. The claim is cleared when the winning turn successfully commits the new head. If a dispatched turn fails or is @@ -422,7 +557,7 @@ Locally, the platform session ID is used unchanged. ### Invocations agent requests and wire compatibility By default, `POST /invocations` accepts `{"message": "Hi", "options": {}, "stream": false}`. Applications can supply -a sync or async `parse_request(request)` returning a typed `InvocationRun(messages, options, stream)` to accept their +a sync or async `parse_input(request)` returning a typed `InvocationRun(messages, options, stream)` to accept their own JSON shape and MAF `Message` inputs. A sync or async `prepare_options(request, options)` hook can filter or replace a **copy** of this turn's caller options without changing the agent's `default_options`. It must return a mapping with string keys. The host rejects reserved platform/session fields, `store`, `extra_body`, and private continuation @@ -470,26 +605,53 @@ applications should use separate state roots or store providers. ### Workflow checkpoints -`ResponsesHostServer` persists workflow checkpoints durably. By default, it uses the -`FoundryCheckpointStore`, backed by Foundry storage when hosted and file-based storage +`ResponsesWorkflowHostServer` and `InvocationsWorkflowHostServer` persist native workflow checkpoints. +By default, they use the `FoundryCheckpointStore`, backed by Foundry storage when hosted and file-based storage locally. Stored checkpoints are scoped under `checkpoints`. ### Function approvals -`ResponsesHostServer` persists function approvals durably. By default, it uses the -`FoundryFunctionApprovalStore`, backed by Foundry storage when hosted and file-based -storage locally. Stored approvals are scoped under `function_approvals`. +The native agent tracks pending approvals, validates decisions, and consumes them +in its `AgentSession`. Hosting persists and restores the complete native session +through `agent_session_store_provider`; it does not maintain a second approval store. +An approval reply must continue the issuing session through `previous_response_id` +or a named conversation. The wire approval ID alone does not establish authority. + +Custom agents must establish matching native pending state by the end of a run +that emits an approval request. Hosting rejects unmatched or unrecorded requests +before publishing an approval item; it does not register approvals on an agent's behalf. + +AgentServer requires its own wire-ID format. The adapter stores only native-ID +and session-ID correlation in the response item's internal metadata, which the SDK +removes from client-facing responses. It resolves that metadata through the scoped +response provider, independently of model-history limits, then delegates binding +and consumption to Core. Native workflow authority remains in its exact checkpoint. + +Remove `function_approval_store_provider` and imports of `FunctionApprovalStore`, +`FoundryFunctionApprovalStore`, and `FunctionApprovalStoreProvider` when upgrading. +Old approval-store records are not migrated. Start a fresh conversation rather +than resuming approvals emitted before this migration. + +Approval output waits for native finalization, and the session is saved before +an approval request is published. This preserves Core's existing session-level +guarantees, not durable single-consumption across +concurrent workers or independently restored old snapshots. Stronger session-store +coordination is separate work. Incomplete hosted MCP batches still need the SDK +preflight support tracked by #9537 to return HTTP 400 before streaming starts; +this migration does not claim that behavior. ## Native Invocations workflows Host a native workflow with an explicit application parser rather than wrapping it -with `workflow.as_agent()`, which is deprecated for hosting (see -[Hosting a `WorkflowAgent` is deprecated](#hosting-a-workflowagent-is-deprecated)): +with `workflow.as_agent()`, which agent hosts reject (see +[Migrate `WorkflowAgent` hosting](#migrate-workflowagent-hosting)): ```python -server = InvocationsHostServer( +from agent_framework_foundry_hosting import CheckpointStoreProvider, InvocationsWorkflowHostServer + +server = InvocationsWorkflowHostServer( workflow=build_workflow, - parse_request=parse_request, + parse_input=parse_input, checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=["my_app:Ticket", "my_app:TicketState"] ), @@ -506,8 +668,8 @@ validates the entire reply batch against its exact pending checkpoint. A sync/async `build_workflow(request)` factory returns a **built**, fresh `Workflow`, with stable name, executor IDs and graph identity. Build fresh executors, agents, clients, providers and tools for each request; allocation must not execute tools or -workflow side effects. Hosted workflows require a factory. Local built instances -are single-use and remain application-owned. Factory-owned agents are entered and +workflow side effects. Both hosted and local workflows require a factory; built instances +are rejected at construction. Factory-owned agents are entered and closed per turn; applications must explicitly own external credentials/providers through their resource-managing agent subclasses. Do not configure another checkpoint store on `WorkflowBuilder` or share mutable graphs between requests. @@ -555,9 +717,9 @@ If a connection drops after commit, retrying the same trusted user/sandbox/call replays that response's complete stored snapshot without executing the workflow or replacing its head. A different call ID starts normal turn validation and cannot retrieve that snapshot. -Local callers may pass one built workflow for a one-shot non-pausing run. Any -workflow that can pause for external input must use a request-aware factory so a -fresh graph can restore and consume the durable reply on the next request. +Local callers also use a request-aware factory, including one-shot non-pausing +runs. Each request receives a fresh graph; a continuation restores and consumes +its durable reply from the exact scoped checkpoint. After a newer Invocations head is durably committed, the host reclaims completed ancestor response records and checkpoints oldest-first, retaining only current, pending, active or blocked authority. Cleanup failures are logged and retried by diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py index cd37bfcf691..cea3f1dc478 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/__init__.py @@ -1,26 +1,32 @@ # Copyright (c) Microsoft. All rights reserved. +"""Expose Foundry agent hosts, native workflow hosts, and scoped storage integrations.""" + +from __future__ import annotations + import importlib.metadata from typing import TYPE_CHECKING, Any, Final if TYPE_CHECKING: - from ._invocations import InvocationsHostServer - from ._request import HostedResponseRequest, InvocationRun, WorkflowTurn, response_input_messages - from ._responses import ResponsesHostServer + from ._invocations_agent import InvocationsHostServer + from ._invocations_workflow import InvocationsWorkflowHostServer + from ._responses_agent import ResponsesHostServer + from ._responses_workflow import ResponsesWorkflowHostServer from ._scope import FoundryRequestScope - from ._state_store import ( + from ._shared import ( AgentSessionStoreProvider, CheckpointStoreProvider, ContextScopedStoreProvider, FoundryAgentSessionStore, FoundryCheckpointStore, - FoundryFunctionApprovalStore, - FunctionApprovalStore, - FunctionApprovalStoreProvider, + InvocationRun, + ResponsesRequest, StoreProvider, + WorkflowSource, + WorkflowTurn, + response_input_messages, ) from ._toolbox import FoundryToolbox - from ._workflow_source import WorkflowSource try: __version__ = importlib.metadata.version(__name__) @@ -28,24 +34,23 @@ __version__ = "0.0.0" _LAZY_EXPORTS: Final[dict[str, str]] = { - "AgentSessionStoreProvider": "._state_store", - "CheckpointStoreProvider": "._state_store", - "ContextScopedStoreProvider": "._state_store", - "FoundryAgentSessionStore": "._state_store", - "FoundryCheckpointStore": "._state_store", - "FoundryFunctionApprovalStore": "._state_store", + "AgentSessionStoreProvider": "._shared", + "CheckpointStoreProvider": "._shared", + "ContextScopedStoreProvider": "._shared", + "FoundryAgentSessionStore": "._shared", + "FoundryCheckpointStore": "._shared", "FoundryRequestScope": "._scope", "FoundryToolbox": "._toolbox", - "HostedResponseRequest": "._request", - "InvocationRun": "._request", - "FunctionApprovalStore": "._state_store", - "FunctionApprovalStoreProvider": "._state_store", - "InvocationsHostServer": "._invocations", - "ResponsesHostServer": "._responses", - "StoreProvider": "._state_store", - "WorkflowSource": "._workflow_source", - "WorkflowTurn": "._request", - "response_input_messages": "._request", + "InvocationRun": "._shared", + "InvocationsHostServer": "._invocations_agent", + "InvocationsWorkflowHostServer": "._invocations_workflow", + "ResponsesHostServer": "._responses_agent", + "ResponsesRequest": "._shared", + "ResponsesWorkflowHostServer": "._responses_workflow", + "StoreProvider": "._shared", + "WorkflowSource": "._shared", + "WorkflowTurn": "._shared", + "response_input_messages": "._shared", } __all__ = [ @@ -54,15 +59,14 @@ "ContextScopedStoreProvider", "FoundryAgentSessionStore", "FoundryCheckpointStore", - "FoundryFunctionApprovalStore", "FoundryRequestScope", "FoundryToolbox", - "FunctionApprovalStore", - "FunctionApprovalStoreProvider", - "HostedResponseRequest", "InvocationRun", "InvocationsHostServer", + "InvocationsWorkflowHostServer", "ResponsesHostServer", + "ResponsesRequest", + "ResponsesWorkflowHostServer", "StoreProvider", "WorkflowSource", "WorkflowTurn", diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_agent_source.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_agent_source.py deleted file mode 100644 index 5f9509b3fef..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_agent_source.py +++ /dev/null @@ -1,39 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from __future__ import annotations - -import inspect -from collections.abc import Awaitable, Callable -from typing import TypeAlias, TypeGuard, cast - -from agent_framework import SupportsAgentRun - -AgentSource: TypeAlias = SupportsAgentRun | Callable[[], SupportsAgentRun | Awaitable[SupportsAgentRun]] - - -def is_agent(value: object) -> TypeGuard[SupportsAgentRun]: - return not inspect.isclass(value) and isinstance(value, SupportsAgentRun) - - -def validate_agent_source(source: object) -> None: - if is_agent(source): - return - if not callable(source): - raise TypeError("agent must be an agent instance or a zero-argument callable that creates one.") - try: - inspect.signature(source).bind() - except (TypeError, ValueError) as exc: - raise TypeError("agent callable must accept no arguments.") from exc - - -async def resolve_agent(source: AgentSource) -> SupportsAgentRun: - """Resolve an agent instance or request-scoped agent factory.""" - if is_agent(source): - return source - - factory = cast(Callable[[], SupportsAgentRun | Awaitable[SupportsAgentRun]], source) - result = factory() - agent = await result if inspect.isawaitable(result) else result - if not is_agent(agent): - raise TypeError("The agent factory must return an object implementing SupportsAgentRun.") - return agent diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations.py deleted file mode 100644 index f0381b1223f..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations.py +++ /dev/null @@ -1,917 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from __future__ import annotations - -import asyncio -import inspect -import json -import logging -import math -import sys -import uuid -import warnings -import weakref -from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping, Sequence -from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager -from contextvars import Token -from copy import deepcopy -from dataclasses import asdict, is_dataclass -from typing import Any, cast - -from agent_framework import ( - AgentResponse, - AgentResponseUpdate, - AgentSession, - ChatResponse, - ChatResponseUpdate, - CheckpointStorage, - ResponseStream, - SessionStore, - SupportsAgentRun, - Workflow, - WorkflowAgent, - WorkflowCheckpointException, - WorkflowEvent, - WorkflowExecutor, -) -from agent_framework._telemetry import mark_feature_used -from anyio import CancelScope -from azure.ai.agentserver.core import ( - FoundryAgentRequestContext, - get_request_context, - reset_request_context, - set_request_context, -) -from azure.ai.agentserver.core.storage import FoundryStorageConflictError, FoundryStoragePreconditionError -from azure.ai.agentserver.invocations import InvocationAgentServerHost -from pydantic import BaseModel -from starlette.requests import Request -from starlette.responses import JSONResponse, Response, StreamingResponse -from starlette.types import Receive, Scope, Send - -from ._agent_source import AgentSource, is_agent, resolve_agent, validate_agent_source -from ._feature_usage import FeatureIndex -from ._request import ( - InvocationRun, - UnsupportedOptions, - WorkflowTurn, - validate_request_options, - validate_unsupported_options, -) -from ._scope import FoundryRequestScope -from ._state_store import AgentSessionStoreProvider, CheckpointStoreProvider, ContextScopedStoreProvider, StoreProvider -from ._workflow_source import WorkflowResolver, WorkflowSource, workflow_agents -from ._workflow_state import HostedWorkflowRun, WorkflowBlockedError, WorkflowConflictError - -logger = logging.getLogger(__name__) - -InvocationParser = Callable[[Request], InvocationRun | WorkflowTurn[Any] | Awaitable[InvocationRun | WorkflowTurn[Any]]] -InvocationOptionsHook = Callable[[Request, dict[str, Any]], Mapping[str, Any] | Awaitable[Mapping[str, Any]]] - -_AGENT_CONTROLLED_FIELDS = frozenset({ - "additional_function_arguments", - "client_kwargs", - "compaction_strategy", - "function_invocation_kwargs", - "instructions", - "middleware", - "session", - "tokenizer", - "tools", -}) -_MAX_WORKFLOW_SNAPSHOT_EVENTS = 4096 -_MAX_WORKFLOW_SNAPSHOT_BYTES = 4 * 1024 * 1024 - - -class _UnsupportedAgentOptions(TypeError): - """The agent cannot accept the caller's run options under the selected policy.""" - - -class _WorkflowOutputError(TypeError): - """Workflow application data cannot be represented as JSON.""" - - -class _WorkflowRequestError(ValueError): - """A validated application request cannot be dispatched to the workflow.""" - - -def _validate_workflow_mapping_keys(value: Any, path: str, active: set[int]) -> None: - if isinstance(value, Mapping): - mapping = cast(Mapping[object, Any], value) - identifier = id(mapping) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") - active.add(identifier) - try: - for index, (key, item) in enumerate(mapping.items()): - if not isinstance(key, str): - raise _WorkflowOutputError( - f"Workflow output at {path} is not JSON-serializable: " - f"mapping keys must be strings, found {type(key).__name__}." - ) - _validate_workflow_mapping_keys(item, f"{path}.value[{index}]", active) - finally: - active.remove(identifier) - elif isinstance(value, (list, tuple)): - sequence = cast(Sequence[Any], value) - identifier = id(sequence) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") - active.add(identifier) - try: - for index, item in enumerate(sequence): - _validate_workflow_mapping_keys(item, f"{path}[{index}]", active) - finally: - active.remove(identifier) - - -def _workflow_serialized_value(value: Any, path: str) -> Any: - if isinstance(value, (AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate)): - return value.to_dict(exclude={"continuation_token"}) - if isinstance(value, BaseModel): - _validate_workflow_mapping_keys(value.model_dump(mode="python"), f"{path}.model", set()) - return value.model_dump(mode="json") - if is_dataclass(value) and not isinstance(value, type): - return asdict(value) - to_dict = getattr(value, "to_dict", None) - if callable(to_dict): - return to_dict() - raise _WorkflowOutputError( - f"Workflow output at {path} is not JSON-serializable: unsupported type {type(value).__name__}." - ) - - -def _workflow_json_value(value: Any, path: str, active: set[int]) -> Any: - if value is None or isinstance(value, (str, int, bool)): - return value - if isinstance(value, float): - if not math.isfinite(value): - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: number must be finite.") - return value - if isinstance(value, Mapping): - mapping = cast(Mapping[object, Any], value) - identifier = id(mapping) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") - active.add(identifier) - try: - result: dict[str, Any] = {} - for index, (key, item) in enumerate(mapping.items()): - if not isinstance(key, str): - raise _WorkflowOutputError( - f"Workflow output at {path} is not JSON-serializable: " - f"mapping keys must be strings, found {type(key).__name__}." - ) - result[key] = _workflow_json_value(item, f"{path}.value[{index}]", active) - return result - finally: - active.remove(identifier) - if isinstance(value, (list, tuple)): - sequence = cast(Sequence[Any], value) - identifier = id(sequence) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") - active.add(identifier) - try: - return [_workflow_json_value(item, f"{path}[{index}]", active) for index, item in enumerate(sequence)] - finally: - active.remove(identifier) - identifier = id(value) - if identifier in active: - raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular serialized value.") - active.add(identifier) - try: - serialized = _workflow_serialized_value(value, path) - return _workflow_json_value(serialized, f"{path}.serialized", active) - finally: - active.remove(identifier) - - -def _workflow_output(value: Any) -> Any: - """Normalize typed application data without text coercion or skipped values.""" - try: - normalized = _workflow_json_value(value, "$", set()) - return json.loads(json.dumps(normalized, allow_nan=False, ensure_ascii=False).encode("utf-8")) - except _WorkflowOutputError: - raise - except (TypeError, ValueError, RecursionError) as exc: - raise _WorkflowOutputError(f"Workflow output type {type(value).__name__} is not JSON-serializable.") from exc - - -def _workflow_frame(event: WorkflowEvent[Any]) -> dict[str, Any] | None: - if event.type not in ("output", "request_info"): - return None - frame: dict[str, Any] = {"type": event.type, "data": event.data} - if event.type == "request_info": - frame["request_id"] = event.request_id - frame["source_executor_id"] = event.source_executor_id - else: - frame["executor_id"] = event.executor_id - return cast(dict[str, Any], _workflow_output(frame)) - - -def _workflow_snapshot_frames(snapshot: Mapping[str, Any] | None) -> list[dict[str, Any]]: - if snapshot is None or set(snapshot) != {"output"} or not isinstance(snapshot["output"], list): - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - frames: list[dict[str, Any]] = [] - for item in cast(list[Any], snapshot["output"]): - if not isinstance(item, Mapping): - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - frame = cast(dict[str, Any], _workflow_output(item)) - if frame.get("type") == "output": - if set(frame) != {"type", "data", "executor_id"}: - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - elif frame.get("type") == "request_info": - if set(frame) != {"type", "data", "request_id", "source_executor_id"}: - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - else: - raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") - frames.append(frame) - return frames - - -def _workflow_frame_size(frame: Mapping[str, Any]) -> int: - return len(json.dumps(frame, ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) - - -def _workflow_can_pause(workflow: Workflow) -> bool: - for executor in workflow.get_executors_list(): - if getattr(executor, "is_request_response_capable", False) is True: - return True - if isinstance(executor, WorkflowExecutor) and _workflow_can_pause(executor.workflow): - return True - return False - - -def _sse(event: str, data: Mapping[str, Any]) -> str: - return f"event: {event}\ndata: {json.dumps(data)}\n\n" - - -def _invocation_failure(exc: Exception) -> tuple[str, int, str]: - cause: BaseException | None = exc - while cause is not None: - if isinstance(cause, (FoundryStorageConflictError, FoundryStoragePreconditionError)): - return "Another request advanced this agent session; reload before retrying.", 409, "session_conflict" - cause = cause.__cause__ - return "Agent invocation failed.", 500, "invocation_failed" - - -def _workflow_failure(exc: Exception) -> tuple[str, int, str]: - if isinstance(exc, _WorkflowOutputError): - return str(exc), 500, "invalid_workflow_output" - cause: BaseException | None = exc - while cause is not None: - if isinstance(cause, WorkflowBlockedError): - return ( - "This workflow turn was interrupted or failed; start a new Foundry sandbox.", - 409, - "workflow_blocked", - ) - if isinstance(cause, WorkflowConflictError): - return str(cause), 409, "workflow_conflict" - cause = cause.__cause__ - if _invocation_failure(exc)[1] == 409: - return "Another request advanced this workflow; reload before retrying.", 409, "workflow_conflict" - if isinstance(exc, _WorkflowRequestError): - return str(exc), 400, "invalid_workflow_turn" - if isinstance(exc, PermissionError): - return "Workflow state is not available in this trusted scope.", 403, "workflow_scope" - return "Workflow invocation failed.", 500, "workflow_invocation_failed" - - -class _InvocationStreamingResponse(StreamingResponse): - def __init__(self, content: AsyncGenerator[str]) -> None: - super().__init__( - content, - media_type="text/event-stream", - headers={"Cache-Control": "no-cache", "Connection": "keep-alive"}, - ) - self._content = content - - async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: - try: - await super().__call__(scope, receive, send) - finally: - # The SDK may wrap the iterator in a keep-alive pump; join it before closing its source. - try: - close = getattr(self.body_iterator, "aclose", None) - if close is not None: - await close() - finally: - if self.body_iterator is not self._content: - await self._content.aclose() - - -class _WorkflowStreamingResponse(_InvocationStreamingResponse): - def __init__(self, content: AsyncGenerator[str], *, keep_alive: bool) -> None: - super().__init__(content) - self._keep_alive = keep_alive - - async def stream_response(self, send: Send) -> None: - try: - await super().stream_response(send) - finally: - with CancelScope(shield=True): - if self._keep_alive and self.body_iterator is not self._content: - close = getattr(self.body_iterator, "aclose", None) - if close is None: - raise TypeError("The Invocations SDK stream iterator must support aclose.") - await close() - try: - await self._content.aclose() - finally: - # Without the SDK pump, finish its wrapper in the task that created its context tokens. - async for _ in self.body_iterator: - pass - - -class InvocationsHostServer(InvocationAgentServerHost): - """Host an agent or native workflow with durable, application-defined Invocations turns.""" - - def __init__( - self, - agent: AgentSource | None = None, - *, - workflow: WorkflowSource[Request] | None = None, - openapi_spec: dict[str, Any] | None = None, - agent_session_store_provider: StoreProvider[SessionStore] | None = None, - checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - parse_request: InvocationParser | None = None, - prepare_options: InvocationOptionsHook | None = None, - unsupported_options: UnsupportedOptions = "warn", - legacy_wire_format: bool = False, - **kwargs: Any, - ) -> None: - """Initialize an InvocationsHostServer. - - Args: - agent: The agent to handle responses for, or a zero-argument sync or async callable that creates one for - each request. Use a callable for agents that keep mutable state outside `AgentSession`. - Hosting a `WorkflowAgent` here is deprecated and should be avoided: it is stateful, so one instance - must never serve requests from different users or conversations. Use `workflow=` instead. - workflow: A built native workflow or a sync/async request-aware factory returning a freshly built - workflow with stable graph/executor IDs. Hosted workflows require a factory. Supply exactly - one of agent or workflow; local workflow instances are single-use and remain application-owned. - openapi_spec: The OpenAPI specification for the server. - agent_session_store_provider: Provider for conversation session storage. Defaults to Foundry storage - when hosted and the SDK's file-backed storage locally. New default stores expire sessions 30 days - after their last write. Custom providers control their own retention. - checkpoint_store_provider: Native workflow checkpoint storage. The default accepts the host's - verified sandbox/user scope. Custom providers must preserve scoped conditional-write semantics. - parse_request: Sync or async parser returning `InvocationRun` for agents or `WorkflowTurn` for - workflows. Native workflows require an explicit parser for typed input or complete pending - reply batches. Without an agent parser, accepts `message`, optional `options` and `stream`. - prepare_options: Optional sync or async hook to filter or replace a copy of caller generation options. - Tool context and agent execution controls must remain in developer-owned agent configuration. - unsupported_options: `"warn"` (default), `"ignore"`, or `"error"` for agents without runtime options. - legacy_wire_format: Opt into the deprecated plain-text response and raw streaming chunks instead of - the default agent JSON response and framed `delta`/`done`/`error` server-sent events. - This agent-only compatibility mode is incompatible with native workflows. - **kwargs: Additional keyword arguments. - """ - if agent is not None and workflow is not None: - raise TypeError("Provide exactly one of agent or workflow.") - if agent is None and workflow is None: - raise TypeError( - "Provide exactly one of agent or workflow; " - "agent must be an agent instance or a zero-argument callable that creates one." - ) - if workflow is None: - validate_agent_source(agent) - elif parse_request is None: - raise TypeError("parse_request is required when hosting a native Workflow.") - if parse_request is not None and not callable(parse_request): - raise TypeError("parse_request must be a callable.") - if prepare_options is not None and not callable(prepare_options): - raise TypeError("prepare_options must be a callable.") - if not isinstance(legacy_wire_format, bool): - raise TypeError("legacy_wire_format must be a boolean.") - if workflow is not None: - if legacy_wire_format: - raise ValueError("legacy_wire_format=True is incompatible with native workflows.") - if prepare_options is not None: - raise ValueError("prepare_options is agent-only; use the workflow parser's explicit client_kwargs.") - if agent_session_store_provider is not None: - raise ValueError( - "agent_session_store_provider is agent-only; use checkpoint_store_provider for workflows." - ) - self._workflow_resolver: WorkflowResolver[Request] | None = ( - WorkflowResolver(workflow) if workflow is not None else None - ) - super().__init__(openapi_spec=openapi_spec, **kwargs) - if self._workflow_resolver is not None and self.config.is_hosted and not self._workflow_resolver.is_factory: - raise ValueError("Hosted native workflows require a request-aware factory that creates fresh executors.") - - self._agent = agent - self._owns_request_agent = agent is not None and not is_agent(agent) - self._warned_workflow_agent = False - self._parse_request = parse_request - self._prepare_options = prepare_options - self._unsupported_options = validate_unsupported_options(unsupported_options) - self._legacy_wire_format = legacy_wire_format - self._session_store_provider = ( - AgentSessionStoreProvider(store_name="invocation_sessions") - if agent_session_store_provider is None - else agent_session_store_provider - ) - self._checkpoint_storage_provider = ( - CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider - ) - self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( - weakref.WeakValueDictionary() - ) - if legacy_wire_format: - message = ( - "legacy_wire_format=True is deprecated; migrate Invocations clients to JSON responses " - "and framed SSE events before removing this compatibility mode." - ) - warnings.warn(message, DeprecationWarning, stacklevel=2) - logger.warning("DEPRECATION: %s", message) - if isinstance(agent, WorkflowAgent): - self._warn_legacy_workflow() - self.invoke_handler(self._handle_invoke) - mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) - - def _partition_key( - self, - *, - context: FoundryAgentRequestContext | None = None, - scope: FoundryRequestScope | None = None, - ) -> str | tuple[str, str]: - """Get the partition key for the current request. - - A hosted partition key is a tuple containing the session ID and user ID, - preserving their boundaries. Locally, the key is just the session ID. In the - Foundry hosted environment, the partition key is used to maintain isolation between - different sessions and users, such that one user cannot access another user's sessions. - - Returns: - The partition key for the current request. - - Exceptions: - RuntimeError: If the context doesn't contain the expected IDs. - """ - if context is None: - context = get_request_context() - - if self.config.is_hosted: - if not context.user_id: - raise RuntimeError( - "The hosted environment is missing user_id in the request context. " - "Please ensure that the request is coming from a valid Foundry platform service." - ) - hosted_scope = scope or FoundryRequestScope.from_context(self.config, context) - return hosted_scope.session_id, context.user_id - - if not context.session_id: - raise RuntimeError( - "The request context is missing session_id. Please ensure that the request is a valid request." - ) - - return context.session_id - - def _hosted_scope(self, request: Request, context: FoundryAgentRequestContext) -> FoundryRequestScope: - """Accept an unconfigured session only when the routed Invocations query identifies it.""" - if not context.user_id: - raise RuntimeError("The hosted environment is missing user_id in the request context.") - - routed_session_ids = request.query_params.getlist("agent_session_id") - if len(routed_session_ids) > 1: - raise RuntimeError("Hosted Invocations requires exactly one agent_session_id query parameter.") - routed_session_id = routed_session_ids[0] if routed_session_ids else None - if self.config.session_id: - if routed_session_id is not None and routed_session_id != self.config.session_id: - raise RuntimeError("The request agent_session_id does not match the platform session ID.") - return FoundryRequestScope.from_context(self.config, context) - - if not routed_session_id or routed_session_id != context.session_id: - raise RuntimeError( - "Hosted Invocations without FOUNDRY_AGENT_SESSION_ID require an explicit routed " - "agent_session_id query parameter matching the request context." - ) - if not context.call_id: - raise RuntimeError("Foundry hosted requests require a trusted user ID and call ID.") - return FoundryRequestScope( - session_id=routed_session_id, - user_id=context.user_id, - call_id=context.call_id, - is_hosted=True, - ) - - def _warn_legacy_workflow(self) -> None: - if not self._warned_workflow_agent: - self._warned_workflow_agent = True - message = ( - "Hosting WorkflowAgent through agent= is deprecated for this beta release and should be avoided. " - "A WorkflowAgent is stateful and keeps workflow state in memory between runs, so one instance must " - "never serve requests from different users or conversations. " - "Use workflow=a_request_aware_factory with an explicit parse_request. " - "A factory that builds a new WorkflowAgent for every request only suits stateless, " - "single-turn workflows, because the Invocations host does not restore workflow checkpoints. " - "Wrapper history, context providers, and event semantics are not automatically unwrapped." - ) - warnings.warn(message, DeprecationWarning, stacklevel=3) - # Request-time calls cannot be attributed to application code, so Python's default filter hides the - # warning there; log it as well. - logger.warning("DEPRECATION: %s", message) - - @asynccontextmanager - async def _request_agent(self) -> AsyncGenerator[SupportsAgentRun]: - if self._agent is None: - raise RuntimeError("No agent is configured for Invocations.") - agent = await resolve_agent(self._agent) - if isinstance(agent, WorkflowAgent): - self._warn_legacy_workflow() - async with AsyncExitStack() as resources: - if self._owns_request_agent and isinstance(agent, AbstractAsyncContextManager): - await resources.enter_async_context(agent) - yield agent - - @asynccontextmanager - async def _request_session( - self, - partition_key: str | tuple[str, str], - context: FoundryAgentRequestContext, - *, - hosted_scope: FoundryRequestScope | None = None, - ) -> AsyncGenerator[AgentSession]: - session_id = ( - json.dumps(partition_key, separators=(",", ":")) if isinstance(partition_key, tuple) else partition_key - ) - try: - provider = self._session_store_provider - if hosted_scope is not None and not self.config.session_id and type(provider) is AgentSessionStoreProvider: - store = provider._get_scoped_store(context, hosted_scope) # pyright: ignore[reportPrivateUsage] - else: - store = provider.get_store(config=self.config, platform_context=context) - session = await store.get(session_id) - if session is None: - session = AgentSession(session_id=session_id) - except Exception: - logger.exception("Failed to load invocation session") - raise - try: - yield session - finally: - failure = sys.exc_info()[1] - try: - await store.set(session_id, session) - except Exception as exc: - logger.exception("Failed to persist invocation session") - if failure is None: - raise - if isinstance(failure, Exception): - raise RuntimeError( - f"Invocation failed: {str(failure) or type(failure).__name__}; " - f"session persistence also failed: {str(exc) or type(exc).__name__}" - ) from exc - - async def _parse(self, request: Request) -> InvocationRun | WorkflowTurn[Any]: - if self._parse_request is not None: - result = self._parse_request(request) - parsed = await result if inspect.isawaitable(result) else result - if self._workflow_resolver is not None: - if not isinstance(parsed, WorkflowTurn): - raise TypeError("parse_request must return WorkflowTurn.") - elif not isinstance(parsed, InvocationRun): - raise TypeError("parse_request must return InvocationRun.") - return parsed - - payload = await request.json() - if not isinstance(payload, dict): - raise ValueError("The invocation must be a JSON object.") - body = cast(Mapping[str, Any], payload) - message = body.get("message") - stream = body.get("stream", False) - options = body.get("options", {}) - if not isinstance(message, str): - raise ValueError("message must be a string.") - if not isinstance(stream, bool): - raise ValueError("stream must be a boolean.") - if not isinstance(options, dict): - raise ValueError("options must be an object.") - return InvocationRun( - messages=message if stream else [message], options=cast(Mapping[str, Any], options), stream=stream - ) - - async def _options(self, request: Request, parsed: InvocationRun) -> dict[str, Any]: - options = deepcopy(dict(parsed.options)) - if self._prepare_options is not None: - result = self._prepare_options(request, options) - if inspect.isawaitable(result): - result = await result - if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): - raise TypeError("prepare_options must return a mapping of MAF run options with string keys.") - options = deepcopy(dict(result)) - validate_request_options(options) - reserved = _AGENT_CONTROLLED_FIELDS.intersection(options) - if reserved: - raise ValueError(f"Invocations options cannot set agent-controlled fields: {', '.join(sorted(reserved))}.") - return options - - def _agent_kwargs(self, agent: SupportsAgentRun, options: dict[str, Any]) -> dict[str, Any]: - if not options: - return {} - try: - inspect.signature(agent.run).bind_partial(options=options) - except (TypeError, ValueError): - if self._unsupported_options == "error": - raise _UnsupportedAgentOptions("The hosted agent does not accept caller runtime options.") from None - if self._unsupported_options == "warn": - logger.warning("Agent doesn't support runtime options. They will be ignored.") - return {} - return {"options": options} - - @staticmethod - async def _close_interrupted_stream(stream: object) -> None: - if isinstance(stream, ResponseStream): - close = getattr(cast(object, stream), "close", None) - if close is None: - logger.warning("The installed core cannot close an interrupted agent stream.") - return - else: - close = getattr(stream, "aclose", None) - if close is not None: - await close() - - async def _handle_invoke(self, request: Request) -> Response: - """Invoke the configured agent or native workflow with a trusted request.""" - context = get_request_context() - try: - hosted_scope = self._hosted_scope(request, context) if self.config.is_hosted else None - partition_key = self._partition_key(context=context, scope=hosted_scope) - except RuntimeError as exc: - logger.error("Failed to resolve Invocations session: %s", exc) - return JSONResponse({"error": str(exc)}, status_code=500) - - try: - parsed = await self._parse(request) - options = await self._options(request, parsed) if isinstance(parsed, InvocationRun) else {} - except (TypeError, ValueError) as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - except Exception: - logger.exception("Failed to prepare Invocations request") - return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) - - if isinstance(parsed, WorkflowTurn): - scope = hosted_scope or FoundryRequestScope.from_context(self.config, context) - return await self._handle_workflow(request, parsed, scope, context, partition_key) - - if parsed.stream: - if options and self._unsupported_options == "error" and is_agent(self._agent): - try: - self._agent_kwargs(self._agent, options) - except _UnsupportedAgentOptions as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - - async def stream_response() -> AsyncGenerator[str]: - token: Token[FoundryAgentRequestContext] | None = set_request_context(context) - try: - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock, self._request_agent() as agent: - run_kwargs = self._agent_kwargs(agent, options) - async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: - stream = agent.run(parsed.messages, session=session, stream=True, **run_kwargs) - completed = False - try: - async for update in stream: - if update.text: - frame = ( - update.text - if self._legacy_wire_format - else _sse("delta", {"text": update.text}) - ) - # The SDK may close a suspended generator from another task. - reset_request_context(token) - token = None - yield frame - token = set_request_context(context) - if isinstance(stream, ResponseStream): - await stream.get_final_response() - completed = True - finally: - if not completed: - if token is None: - token = set_request_context(context) - try: - await self._close_interrupted_stream(stream) - except Exception: - logger.exception("Failed to close interrupted Invocations agent stream") - if not self._legacy_wire_format: - session_id = partition_key[0] if isinstance(partition_key, tuple) else partition_key - if token is not None: - reset_request_context(token) - token = None - yield _sse("done", {"session_id": session_id}) - except _UnsupportedAgentOptions as exc: - if self._legacy_wire_format: - raise - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": str(exc), "code": "unsupported_options", "status": 400}) - except Exception as exc: - logger.exception("Invocations agent stream failed") - if self._legacy_wire_format: - raise - message, status, code = _invocation_failure(exc) - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": message, "code": code, "status": status}) - finally: - if token is not None: - reset_request_context(token) - - return _InvocationStreamingResponse(stream_response()) - - try: - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock, self._request_agent() as agent: - run_kwargs = self._agent_kwargs(agent, options) - async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: - response = await agent.run(parsed.messages, session=session, **run_kwargs) - if self._legacy_wire_format: - return Response(content=response.text) - return JSONResponse({"response": response.text}) - except _UnsupportedAgentOptions as exc: - return JSONResponse({"error": str(exc)}, status_code=400) - except Exception as exc: - logger.exception("Invocations agent request failed") - message, status, _ = _invocation_failure(exc) - return JSONResponse({"error": message}, status_code=status) - - async def _handle_workflow( - self, - request: Request, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - context: FoundryAgentRequestContext, - partition_key: str | tuple[str, str], - ) -> Response: - if turn.stream: - - async def stream_response() -> AsyncGenerator[str]: - token: Token[FoundryAgentRequestContext] | None = set_request_context(context) - events = self._workflow_events(request, turn, scope, context, partition_key) - try: - async for frame in events: - wire_frame = _sse(frame["type"], frame) - reset_request_context(token) - token = None - yield wire_frame - token = set_request_context(context) - reset_request_context(token) - token = None - yield _sse("done", {"session_id": scope.session_id}) - except Exception as exc: - logger.exception("Invocations workflow stream failed") - message, status, code = _workflow_failure(exc) - if token is not None: - reset_request_context(token) - token = None - yield _sse("error", {"message": message, "code": code, "status": status}) - finally: - if token is None: - token = set_request_context(context) - try: - with CancelScope(shield=True): - await events.aclose() - finally: - reset_request_context(token) - - return _WorkflowStreamingResponse(stream_response(), keep_alive=self.config.sse_keepalive_interval > 0) - - try: - frames = [frame async for frame in self._workflow_events(request, turn, scope, context, partition_key)] - return JSONResponse({"output": frames}) - except Exception as exc: - logger.exception("Invocations workflow request failed") - message, status, _ = _workflow_failure(exc) - return JSONResponse({"error": message}, status_code=status) - - async def _workflow_events( - self, - request: Request, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - context: FoundryAgentRequestContext, - partition_key: str | tuple[str, str], - ) -> AsyncGenerator[dict[str, Any]]: - resolver = self._workflow_resolver - if resolver is None: - raise RuntimeError("No native workflow is configured for Invocations.") - frames: list[dict[str, Any]] = [] - snapshot_bytes = 0 - deferred: list[dict[str, Any]] = [] - defer = False - lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) - async with lock: - workflow = await resolver.resolve(request) - if not resolver.is_factory and _workflow_can_pause(workflow): - raise _WorkflowRequestError("A pausing native workflow requires a fresh request-aware factory.") - run = await HostedWorkflowRun.prepare( - workflow, - scope=scope, - response_id=context.call_id or uuid.uuid4().hex, - config=self.config, - platform_context=context, - checkpoint_store_provider=self._checkpoint_storage_provider, - lineage_id="invocations", - stored=resolver.is_factory, - replay_completed=True, - fresh_factory=resolver.is_factory, - reclaim_superseded=True, - ) - if run.has_completed_output: - for frame in _workflow_snapshot_frames(run.snapshot): - yield frame - return - try: - turn = run.validate_turn(turn) - except (TypeError, ValueError) as exc: - raise _WorkflowRequestError(str(exc)) from exc - if ( - run.stored - and turn.input is not None - and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) - ): - try: - self._checkpoint_storage_provider.validate_checkpoint_value(turn.input) - except WorkflowCheckpointException as exc: - raise _WorkflowRequestError( - "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " - 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' - ) from exc - if ( - run.stored - and turn.responses is not None - and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) - ): - try: - for reply in turn.responses.values(): - self._checkpoint_storage_provider.validate_checkpoint_value(reply) - except WorkflowCheckpointException as exc: - raise _WorkflowRequestError( - "Stored native workflow reply uses an application type that is not checkpoint-allowlisted. " - 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' - ) from exc - committed = False - resources = AsyncExitStack() - try: - await run.claim() - try: - if resolver.is_factory: - entered: set[int] = set() - for agent in workflow_agents(workflow): - if id(agent) not in entered and isinstance(agent, AbstractAsyncContextManager): - entered.add(id(agent)) - await resources.enter_async_context(agent) - events = run.events(turn) - resources.push_async_callback(events.aclose) - async for event in events: - if event.type == "request_info" and not resolver.is_factory: - raise _WorkflowRequestError( - "Native workflow pauses require a request-aware factory that rebuilds the workflow." - ) - frame = _workflow_frame(event) - if frame is None: - continue - frame_size = _workflow_frame_size(frame) - if ( - len(frames) >= _MAX_WORKFLOW_SNAPSHOT_EVENTS - or snapshot_bytes + frame_size > _MAX_WORKFLOW_SNAPSHOT_BYTES - ): - raise _WorkflowOutputError( - "Workflow output exceeds the bounded Invocations snapshot limit." - ) - snapshot_bytes += frame_size - frames.append(frame) - defer = defer or event.type == "request_info" - if defer: - deferred.append(frame) - else: - yield frame - finally: - # Starlette cancellation must not interrupt core/resource cleanup. - with CancelScope(shield=True): - await resources.__aexit__(*sys.exc_info()) - await run.commit({"output": frames}) - committed = True - finally: - if not committed: - failure = sys.exc_info()[1] - try: - with CancelScope(shield=True): - await run.abort() - except Exception as exc: - logger.exception("Failed to persist interrupted Invocations workflow state") - if failure is None: - raise - if isinstance(failure, Exception): - raise RuntimeError( - "Workflow invocation failed and its interrupted state could not be persisted." - ) from exc - # Pending reply authority is durable, and a reply need not wait on this stream's lock. - for frame in deferred: - yield frame diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py new file mode 100644 index 00000000000..b38270e56ab --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_agent.py @@ -0,0 +1,384 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Host session-owned agents through the Invocations protocol.""" + +from __future__ import annotations + +import asyncio +import inspect +import json +import logging +import sys +import warnings +import weakref +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping +from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager +from contextvars import Token +from copy import deepcopy +from typing import Any, Literal, cast + +from agent_framework import AgentSession, ResponseStream, SessionStore, SupportsAgentRun +from agent_framework._telemetry import mark_feature_used +from azure.ai.agentserver.core import ( + FoundryAgentRequestContext, + get_request_context, + reset_request_context, + set_request_context, +) +from azure.ai.agentserver.invocations import InvocationAgentServerHost +from starlette.requests import Request +from starlette.responses import JSONResponse, Response + +from ._feature_usage import FeatureIndex +from ._invocations_shared import ( + InvocationOptionsHook, + _invocation_failure, # pyright: ignore[reportPrivateUsage] + _InvocationStreamingResponse, # pyright: ignore[reportPrivateUsage] + _sse, # pyright: ignore[reportPrivateUsage] + invocation_partition_key, + invocation_request_scope, +) +from ._scope import FoundryRequestScope +from ._shared import ( + AgentSessionStoreProvider, + AgentSource, + InvocationRun, + StoreProvider, + UnsupportedOptions, + is_agent, + resolve_agent, + validate_agent_source, + validate_request_options, + validate_sse_keep_alive_interval, + validate_unsupported_options, +) + +logger = logging.getLogger(__name__) + +_AGENT_CONTROLLED_FIELDS = frozenset({ + "additional_function_arguments", + "client_kwargs", + "compaction_strategy", + "function_invocation_kwargs", + "instructions", + "middleware", + "session", + "tokenizer", + "tools", +}) + + +class _UnsupportedAgentOptions(TypeError): + """The agent cannot accept the caller's run options under the selected policy.""" + + +class InvocationsHostServer(InvocationAgentServerHost): + """Host an agent with application-defined Invocations turns.""" + + def __init__( + self, + agent: AgentSource[Request], + *, + openapi_spec: dict[str, Any] | None = None, + agent_session_store_provider: StoreProvider[SessionStore] | None = None, + parse_input: Callable[[Request], InvocationRun | Awaitable[InvocationRun]] | None = None, + prepare_options: InvocationOptionsHook | None = None, + unsupported_options: UnsupportedOptions = "warn", + legacy_wire_format: bool = False, + sse_keep_alive_interval_seconds: int | None = None, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, + ) -> None: + """Create an agent host with session-backed application turns. + + Args: + agent: Agent instance or sync/async factory called once per request. Factories must declare a + required parameter named `request`, accepting the Starlette `Request` passed by keyword. + Extra parameters must have defaults. The factory may ignore the request. + Factories run after request parsing and option preparation. Use a factory when mutable state + is not session-scoped. The host enters and closes factory-owned + context-managed agents; injected instances remain application-owned. `WorkflowAgent` is + rejected; use `InvocationsWorkflowHostServer` with a native workflow factory instead. + + Keyword Args: + openapi_spec: Application OpenAPI specification forwarded to AgentServer. + agent_session_store_provider: Conversation session storage. Defaults to Foundry storage when + hosted and file-backed storage locally. Default stores expire 30 days after the last write. + parse_input: Sync or async parser returning `InvocationRun`. Without a parser, accept JSON + `message`, optional `options`, and `stream`. + prepare_options: Sync or async hook receiving the request and a copy of caller generation + options. Return the replacement mapping; hosting validates it before agent dispatch. + unsupported_options: Policy for agents without runtime options: `"warn"`, `"ignore"`, or `"error"`. + legacy_wire_format: Deprecated agent-only plain-text and raw-stream compatibility format. + Heartbeats are disabled in this format. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + An explicit interval cannot be combined with `legacy_wire_format=True`. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. + + Raises: + TypeError: If the agent source, parser/hook, heartbeat type, or legacy-format flag is invalid. + ValueError: If the heartbeat interval is nonpositive, raw legacy output requests heartbeats, + or the unsupported-options policy is invalid. + """ + validate_agent_source(agent) + if parse_input is not None and not callable(parse_input): + raise TypeError("parse_input must be a callable.") + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + if not isinstance(legacy_wire_format, bool): + raise TypeError("legacy_wire_format must be a boolean.") + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + if legacy_wire_format and sse_keep_alive_interval_seconds is not None: + raise ValueError("SSE heartbeats require framed SSE; disable legacy_wire_format.") + unsupported_options = validate_unsupported_options(unsupported_options) + super().__init__( + openapi_spec=openapi_spec, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) + if legacy_wire_format: + self.config.sse_keepalive_interval = 0 + elif sse_keep_alive_interval_seconds is not None: + self.config.sse_keepalive_interval = sse_keep_alive_interval_seconds + self._agent = agent + self._owns_request_agent = not is_agent(agent) + self._parse_input = parse_input + self._prepare_options = prepare_options + self._unsupported_options = unsupported_options + self._legacy_wire_format = legacy_wire_format + self._session_store_provider = ( + AgentSessionStoreProvider(store_name="invocation_sessions") + if agent_session_store_provider is None + else agent_session_store_provider + ) + self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( + weakref.WeakValueDictionary() + ) + if legacy_wire_format: + message = ( + "legacy_wire_format=True is deprecated; migrate Invocations clients to JSON responses " + "and framed SSE events before removing this compatibility mode." + ) + warnings.warn(message, DeprecationWarning, stacklevel=2) + logger.warning("DEPRECATION: %s", message) + self.invoke_handler(self._handle_invoke) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + @asynccontextmanager + async def _request_agent(self, request: Request) -> AsyncGenerator[SupportsAgentRun]: + agent = await resolve_agent(self._agent, request) + async with AsyncExitStack() as resources: + if self._owns_request_agent and isinstance(agent, AbstractAsyncContextManager): + await resources.enter_async_context(agent) + yield agent + + @asynccontextmanager + async def _request_session( + self, + partition_key: str | tuple[str, str], + context: FoundryAgentRequestContext, + *, + hosted_scope: FoundryRequestScope | None = None, + ) -> AsyncGenerator[AgentSession]: + session_id = ( + json.dumps(partition_key, separators=(",", ":")) if isinstance(partition_key, tuple) else partition_key + ) + try: + provider = self._session_store_provider + if hosted_scope is not None and not self.config.session_id and type(provider) is AgentSessionStoreProvider: + store = provider._get_scoped_store(context, hosted_scope) # pyright: ignore[reportPrivateUsage] + else: + store = provider.get_store(config=self.config, platform_context=context) + session = await store.get(session_id) + if session is None: + session = AgentSession(session_id=session_id) + except Exception: + logger.exception("Failed to load invocation session") + raise + try: + yield session + finally: + failure = sys.exc_info()[1] + try: + await store.set(session_id, session) + except Exception as exc: + logger.exception("Failed to persist invocation session") + if failure is None: + raise + if isinstance(failure, Exception): + raise RuntimeError( + f"Invocation failed: {str(failure) or type(failure).__name__}; " + f"session persistence also failed: {str(exc) or type(exc).__name__}" + ) from exc + + async def _parse(self, request: Request) -> InvocationRun: + if self._parse_input is not None: + result = self._parse_input(request) + parsed = await result if inspect.isawaitable(result) else result + if not isinstance(parsed, InvocationRun): + raise TypeError("parse_input must return InvocationRun.") + return parsed + + payload = await request.json() + if not isinstance(payload, dict): + raise ValueError("The invocation must be a JSON object.") + body = cast(Mapping[str, Any], payload) + message = body.get("message") + stream = body.get("stream", False) + options = body.get("options", {}) + if not isinstance(message, str): + raise ValueError("message must be a string.") + if not isinstance(stream, bool): + raise ValueError("stream must be a boolean.") + if not isinstance(options, dict): + raise ValueError("options must be an object.") + return InvocationRun( + messages=message if stream else [message], options=cast(Mapping[str, Any], options), stream=stream + ) + + async def _options(self, request: Request, parsed: InvocationRun) -> dict[str, Any]: + options = deepcopy(dict(parsed.options)) + if self._prepare_options is not None: + result = self._prepare_options(request, options) + if inspect.isawaitable(result): + result = await result + if not isinstance(result, Mapping) or any(not isinstance(key, str) for key in result): + raise TypeError("prepare_options must return a mapping of MAF run options with string keys.") + options = deepcopy(dict(result)) + validate_request_options(options) + reserved = _AGENT_CONTROLLED_FIELDS.intersection(options) + if reserved: + raise ValueError(f"Invocations options cannot set agent-controlled fields: {', '.join(sorted(reserved))}.") + return options + + def _agent_kwargs(self, agent: SupportsAgentRun, options: dict[str, Any]) -> dict[str, Any]: + if not options: + return {} + try: + inspect.signature(agent.run).bind_partial(options=options) + except (TypeError, ValueError): + if self._unsupported_options == "error": + raise _UnsupportedAgentOptions("The hosted agent does not accept caller runtime options.") from None + if self._unsupported_options == "warn": + logger.warning("Agent doesn't support runtime options. They will be ignored.") + return {} + return {"options": options} + + @staticmethod + async def _close_interrupted_stream(stream: object) -> None: + if isinstance(stream, ResponseStream): + close = getattr(cast(object, stream), "close", None) + if close is None: + logger.warning("The installed core cannot close an interrupted agent stream.") + return + else: + close = getattr(stream, "aclose", None) + if close is not None: + await close() + + async def _handle_invoke(self, request: Request) -> Response: + """Invoke the configured agent with a trusted request.""" + context = get_request_context() + try: + hosted_scope = invocation_request_scope(self.config, request, context) + partition_key = invocation_partition_key(self.config, context=context, scope=hosted_scope) + except RuntimeError as exc: + logger.error("Failed to resolve Invocations session: %s", exc) + return JSONResponse({"error": str(exc)}, status_code=500) + + try: + parsed = await self._parse(request) + options = await self._options(request, parsed) + except (TypeError, ValueError) as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception: + logger.exception("Failed to prepare Invocations request") + return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) + + if parsed.stream: + if options and self._unsupported_options == "error" and is_agent(self._agent): + try: + self._agent_kwargs(self._agent, options) + except _UnsupportedAgentOptions as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + + async def stream_response() -> AsyncGenerator[str]: + token: Token[FoundryAgentRequestContext] | None = set_request_context(context) + try: + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock, self._request_agent(request) as agent: + run_kwargs = self._agent_kwargs(agent, options) + async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: + stream = agent.run(parsed.messages, session=session, stream=True, **run_kwargs) + completed = False + try: + async for update in stream: + if update.text: + frame = ( + update.text + if self._legacy_wire_format + else _sse("delta", {"text": update.text}) + ) + # The SDK may close a suspended generator from another task. + reset_request_context(token) + token = None + yield frame + token = set_request_context(context) + if isinstance(stream, ResponseStream): + await stream.get_final_response() + completed = True + finally: + if not completed: + if token is None: + token = set_request_context(context) + try: + await self._close_interrupted_stream(stream) + except Exception: + logger.exception("Failed to close interrupted Invocations agent stream") + if not self._legacy_wire_format: + session_id = partition_key[0] if isinstance(partition_key, tuple) else partition_key + if token is not None: + reset_request_context(token) + token = None + yield _sse("done", {"session_id": session_id}) + except _UnsupportedAgentOptions as exc: + if self._legacy_wire_format: + raise + if token is not None: + reset_request_context(token) + token = None + yield _sse("error", {"message": str(exc), "code": "unsupported_options", "status": 400}) + except Exception as exc: + logger.exception("Invocations agent stream failed") + if self._legacy_wire_format: + raise + message, status, code = _invocation_failure(exc) + if token is not None: + reset_request_context(token) + token = None + yield _sse("error", {"message": message, "code": code, "status": status}) + finally: + if token is not None: + reset_request_context(token) + + return _InvocationStreamingResponse(stream_response()) + + try: + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock, self._request_agent(request) as agent: + run_kwargs = self._agent_kwargs(agent, options) + async with self._request_session(partition_key, context, hosted_scope=hosted_scope) as session: + response = await agent.run(parsed.messages, session=session, **run_kwargs) + if self._legacy_wire_format: + return Response(content=response.text) + return JSONResponse({"response": response.text}) + except _UnsupportedAgentOptions as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception as exc: + logger.exception("Invocations agent request failed") + message, status, _ = _invocation_failure(exc) + return JSONResponse({"error": message}, status_code=status) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py new file mode 100644 index 00000000000..086082c6bc7 --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_shared.py @@ -0,0 +1,323 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Share Invocations identity, wire conversion, and streaming cleanup helpers.""" + +from __future__ import annotations + +import json +import math +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping, Sequence +from dataclasses import asdict, is_dataclass +from typing import Any, cast + +from agent_framework import AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate, WorkflowEvent +from anyio import CancelScope +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext, get_request_context +from azure.ai.agentserver.core.storage import FoundryStorageConflictError, FoundryStoragePreconditionError +from pydantic import BaseModel +from starlette.requests import Request +from starlette.responses import StreamingResponse +from starlette.types import Receive, Scope, Send + +from ._scope import FoundryRequestScope +from ._shared import WorkflowBlockedError, WorkflowConflictError + +InvocationOptionsHook = Callable[[Request, dict[str, Any]], Mapping[str, Any] | Awaitable[Mapping[str, Any]]] + + +def invocation_partition_key( + config: AgentConfig, + *, + context: FoundryAgentRequestContext | None = None, + scope: FoundryRequestScope | None = None, +) -> str | tuple[str, str]: + """Resolve the session key, preserving hosted session/user boundaries.""" + if context is None: + context = get_request_context() + + if config.is_hosted: + if not context.user_id: + raise RuntimeError( + "The hosted environment is missing user_id in the request context. " + "Please ensure that the request is coming from a valid Foundry platform service." + ) + hosted_scope = scope or FoundryRequestScope.from_context(config, context) + return hosted_scope.session_id, context.user_id + + if not context.session_id: + raise RuntimeError( + "The request context is missing session_id. Please ensure that the request is a valid request." + ) + return context.session_id + + +def invocation_request_scope( + config: AgentConfig, request: Request, context: FoundryAgentRequestContext +) -> FoundryRequestScope | None: + """Resolve hosted identity, requiring a routed query when the platform session is unconfigured.""" + if not config.is_hosted: + return None + if not context.user_id: + raise RuntimeError("The hosted environment is missing user_id in the request context.") + + routed_session_ids = request.query_params.getlist("agent_session_id") + if len(routed_session_ids) > 1: + raise RuntimeError("Hosted Invocations requires exactly one agent_session_id query parameter.") + routed_session_id = routed_session_ids[0] if routed_session_ids else None + if config.session_id: + if routed_session_id is not None and routed_session_id != config.session_id: + raise RuntimeError("The request agent_session_id does not match the platform session ID.") + return FoundryRequestScope.from_context(config, context) + + if not routed_session_id or routed_session_id != context.session_id: + raise RuntimeError( + "Hosted Invocations without FOUNDRY_AGENT_SESSION_ID require an explicit routed " + "agent_session_id query parameter matching the request context." + ) + if not context.call_id: + raise RuntimeError("Foundry hosted requests require a trusted user ID and call ID.") + return FoundryRequestScope( + session_id=routed_session_id, + user_id=context.user_id, + call_id=context.call_id, + is_hosted=True, + ) + + +class _WorkflowOutputError(TypeError): + """Workflow application data cannot be represented as JSON.""" + + +class _WorkflowRequestError(ValueError): + """A validated application request cannot be dispatched to the workflow.""" + + +def _validate_workflow_mapping_keys(value: Any, path: str, active: set[int]) -> None: + if isinstance(value, Mapping): + mapping = cast(Mapping[object, Any], value) + identifier = id(mapping) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") + active.add(identifier) + try: + for index, (key, item) in enumerate(mapping.items()): + if not isinstance(key, str): + raise _WorkflowOutputError( + f"Workflow output at {path} is not JSON-serializable: " + f"mapping keys must be strings, found {type(key).__name__}." + ) + _validate_workflow_mapping_keys(item, f"{path}.value[{index}]", active) + finally: + active.remove(identifier) + elif isinstance(value, (list, tuple)): + sequence = cast(Sequence[Any], value) + identifier = id(sequence) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") + active.add(identifier) + try: + for index, item in enumerate(sequence): + _validate_workflow_mapping_keys(item, f"{path}[{index}]", active) + finally: + active.remove(identifier) + + +def _workflow_serialized_value(value: Any, path: str) -> Any: + if isinstance(value, (AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate)): + return value.to_dict(exclude={"continuation_token"}) + if isinstance(value, BaseModel): + _validate_workflow_mapping_keys(value.model_dump(mode="python"), f"{path}.model", set()) + return value.model_dump(mode="json") + if is_dataclass(value) and not isinstance(value, type): + return asdict(value) + to_dict = getattr(value, "to_dict", None) + if callable(to_dict): + return to_dict() + raise _WorkflowOutputError( + f"Workflow output at {path} is not JSON-serializable: unsupported type {type(value).__name__}." + ) + + +def _workflow_json_value(value: Any, path: str, active: set[int]) -> Any: + if value is None or isinstance(value, (str, int, bool)): + return value + if isinstance(value, float): + if not math.isfinite(value): + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: number must be finite.") + return value + if isinstance(value, Mapping): + mapping = cast(Mapping[object, Any], value) + identifier = id(mapping) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular mapping.") + active.add(identifier) + try: + result: dict[str, Any] = {} + for index, (key, item) in enumerate(mapping.items()): + if not isinstance(key, str): + raise _WorkflowOutputError( + f"Workflow output at {path} is not JSON-serializable: " + f"mapping keys must be strings, found {type(key).__name__}." + ) + result[key] = _workflow_json_value(item, f"{path}.value[{index}]", active) + return result + finally: + active.remove(identifier) + if isinstance(value, (list, tuple)): + sequence = cast(Sequence[Any], value) + identifier = id(sequence) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular sequence.") + active.add(identifier) + try: + return [_workflow_json_value(item, f"{path}[{index}]", active) for index, item in enumerate(sequence)] + finally: + active.remove(identifier) + identifier = id(value) + if identifier in active: + raise _WorkflowOutputError(f"Workflow output at {path} is not JSON-serializable: circular serialized value.") + active.add(identifier) + try: + serialized = _workflow_serialized_value(value, path) + return _workflow_json_value(serialized, f"{path}.serialized", active) + finally: + active.remove(identifier) + + +def _workflow_output(value: Any) -> Any: + """Normalize typed application data without text coercion or skipped values.""" + try: + normalized = _workflow_json_value(value, "$", set()) + return json.loads(json.dumps(normalized, allow_nan=False, ensure_ascii=False).encode("utf-8")) + except _WorkflowOutputError: + raise + except (TypeError, ValueError, RecursionError) as exc: + raise _WorkflowOutputError(f"Workflow output type {type(value).__name__} is not JSON-serializable.") from exc + + +def _workflow_frame( # pyright: ignore[reportUnusedFunction] + event: WorkflowEvent[Any], +) -> dict[str, Any] | None: + if event.type not in ("output", "request_info"): + return None + frame: dict[str, Any] = {"type": event.type, "data": event.data} + if event.type == "request_info": + frame["request_id"] = event.request_id + frame["source_executor_id"] = event.source_executor_id + else: + frame["executor_id"] = event.executor_id + return cast(dict[str, Any], _workflow_output(frame)) + + +def _workflow_snapshot_frames( # pyright: ignore[reportUnusedFunction] + snapshot: Mapping[str, Any] | None, +) -> list[dict[str, Any]]: + if snapshot is None or set(snapshot) != {"output"} or not isinstance(snapshot["output"], list): + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + frames: list[dict[str, Any]] = [] + for item in cast(list[Any], snapshot["output"]): + if not isinstance(item, Mapping): + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + frame = cast(dict[str, Any], _workflow_output(item)) + if frame.get("type") == "output": + if set(frame) != {"type", "data", "executor_id"}: + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + elif frame.get("type") == "request_info": + if set(frame) != {"type", "data", "request_id", "source_executor_id"}: + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + else: + raise RuntimeError("The stored Invocations workflow output snapshot is invalid.") + frames.append(frame) + return frames + + +def _workflow_frame_size( # pyright: ignore[reportUnusedFunction] + frame: Mapping[str, Any], +) -> int: + return len(json.dumps(frame, ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) + + +def _sse( # pyright: ignore[reportUnusedFunction] + event: str, data: Mapping[str, Any] +) -> str: + return f"event: {event}\ndata: {json.dumps(data)}\n\n" + + +def _invocation_failure(exc: Exception) -> tuple[str, int, str]: + cause: BaseException | None = exc + while cause is not None: + if isinstance(cause, (FoundryStorageConflictError, FoundryStoragePreconditionError)): + return "Another request advanced this agent session; reload before retrying.", 409, "session_conflict" + cause = cause.__cause__ + return "Agent invocation failed.", 500, "invocation_failed" + + +def _workflow_failure( # pyright: ignore[reportUnusedFunction] + exc: Exception, +) -> tuple[str, int, str]: + if isinstance(exc, _WorkflowOutputError): + return str(exc), 500, "invalid_workflow_output" + cause: BaseException | None = exc + while cause is not None: + if isinstance(cause, WorkflowBlockedError): + return ( + "This workflow turn was interrupted or failed; start a new Foundry sandbox.", + 409, + "workflow_blocked", + ) + if isinstance(cause, WorkflowConflictError): + return str(cause), 409, "workflow_conflict" + cause = cause.__cause__ + if _invocation_failure(exc)[1] == 409: + return "Another request advanced this workflow; reload before retrying.", 409, "workflow_conflict" + if isinstance(exc, _WorkflowRequestError): + return str(exc), 400, "invalid_workflow_turn" + if isinstance(exc, PermissionError): + return "Workflow state is not available in this trusted scope.", 403, "workflow_scope" + return "Workflow invocation failed.", 500, "workflow_invocation_failed" + + +class _InvocationStreamingResponse(StreamingResponse): + def __init__(self, content: AsyncGenerator[str]) -> None: + super().__init__( + content, + media_type="text/event-stream", + headers={"Cache-Control": "no-cache", "Connection": "keep-alive"}, + ) + self._content = content + + async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: + try: + await super().__call__(scope, receive, send) + finally: + # The SDK may wrap the iterator in a keep-alive pump; join it before closing its source. + try: + close = getattr(self.body_iterator, "aclose", None) + if close is not None: + await close() + finally: + if self.body_iterator is not self._content: + await self._content.aclose() + + +class _WorkflowStreamingResponse(_InvocationStreamingResponse): # pyright: ignore[reportUnusedClass] + def __init__(self, content: AsyncGenerator[str], *, keep_alive: bool) -> None: + super().__init__(content) + self._keep_alive = keep_alive + + async def stream_response(self, send: Send) -> None: + try: + await super().stream_response(send) + finally: + with CancelScope(shield=True): + if self._keep_alive and self.body_iterator is not self._content: + close = getattr(self.body_iterator, "aclose", None) + if close is None: + raise TypeError("The Invocations SDK stream iterator must support aclose.") + await close() + try: + await self._content.aclose() + finally: + # Without the SDK pump, finish its wrapper in the task that created its context tokens. + async for _ in self.body_iterator: + pass diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py new file mode 100644 index 00000000000..11819b8b416 --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations_workflow.py @@ -0,0 +1,307 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Host checkpoint-owned native workflows through the Invocations protocol.""" + +from __future__ import annotations + +import asyncio +import inspect +import logging +import sys +import uuid +import weakref +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping +from contextlib import AbstractAsyncContextManager, AsyncExitStack +from contextvars import Token +from typing import Any, Literal + +from agent_framework import CheckpointStorage, WorkflowCheckpointException +from agent_framework._telemetry import mark_feature_used +from anyio import CancelScope +from azure.ai.agentserver.core import ( + FoundryAgentRequestContext, + get_request_context, + reset_request_context, + set_request_context, +) +from azure.ai.agentserver.invocations import InvocationAgentServerHost +from starlette.requests import Request +from starlette.responses import JSONResponse, Response + +from ._feature_usage import FeatureIndex +from ._invocations_shared import ( + _sse, # pyright: ignore[reportPrivateUsage] + _workflow_failure, # pyright: ignore[reportPrivateUsage] + _workflow_frame, # pyright: ignore[reportPrivateUsage] + _workflow_frame_size, # pyright: ignore[reportPrivateUsage] + _workflow_snapshot_frames, # pyright: ignore[reportPrivateUsage] + _WorkflowOutputError, # pyright: ignore[reportPrivateUsage] + _WorkflowRequestError, # pyright: ignore[reportPrivateUsage] + _WorkflowStreamingResponse, # pyright: ignore[reportPrivateUsage] + invocation_partition_key, + invocation_request_scope, +) +from ._scope import FoundryRequestScope +from ._shared import ( + CheckpointStoreProvider, + ContextScopedStoreProvider, + HostedWorkflowRun, + WorkflowResolver, + WorkflowSource, + WorkflowTurn, + validate_sse_keep_alive_interval, + workflow_agents, +) + +logger = logging.getLogger(__name__) + +_MAX_WORKFLOW_SNAPSHOT_EVENTS = 4096 +_MAX_WORKFLOW_SNAPSHOT_BYTES = 4 * 1024 * 1024 + + +class InvocationsWorkflowHostServer(InvocationAgentServerHost): + """Host a native, checkpoint-owned workflow with application-defined Invocations turns.""" + + def __init__( + self, + workflow: WorkflowSource[Request], + *, + parse_input: Callable[[Request], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], + openapi_spec: dict[str, Any] | None = None, + checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, + sse_keep_alive_interval_seconds: int | None = None, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, + ) -> None: + """Create a native workflow host with explicit parsing and checkpoint ownership. + + Args: + workflow: Sync or async request-aware factory, called for each request. Build fresh workflows, + executors, agents, clients, providers, and mutable tools inside the factory. Keep workflow + and executor IDs stable for checkpoint restoration. The host enters and closes context-managed + agents; applications own other resource cleanup. Known shared execution resources are rejected. + Construct tool objects inside the factory. A module-level `@tool` creates a shared wrapper, + which the ownership guard rejects even for a pure function. A function defined elsewhere can + be wrapped with `tool(function)` inside the factory; applications still own any captured state. + + Keyword Args: + parse_input: Sync or async parser returning typed `WorkflowTurn` input or a complete pending + reply batch. The application validates the request and explicitly supplies workflow kwargs. + openapi_spec: Application OpenAPI specification forwarded to AgentServer. + checkpoint_store_provider: Scoped checkpoint storage. Defaults to `CheckpointStoreProvider`. + Native workflow history and pending replies are checkpoint-owned, not agent-session-owned. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. + + Raises: + TypeError: If the workflow is not a request-aware factory, the parser is invalid, + or the heartbeat type is invalid. + ValueError: If the heartbeat interval is nonpositive. + """ + self._workflow_resolver = WorkflowResolver(workflow) + if not callable(parse_input): + raise TypeError("parse_input must be a callable.") + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + super().__init__( + openapi_spec=openapi_spec, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) + if sse_keep_alive_interval_seconds is not None: + self.config.sse_keepalive_interval = sse_keep_alive_interval_seconds + self._parse_input = parse_input + self._checkpoint_storage_provider = ( + CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider + ) + self._session_locks: weakref.WeakValueDictionary[str | tuple[str, str], asyncio.Lock] = ( + weakref.WeakValueDictionary() + ) + self.invoke_handler(self._handle_invoke) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + async def _parse(self, request: Request) -> WorkflowTurn[Any]: + result = self._parse_input(request) + parsed = await result if inspect.isawaitable(result) else result + if not isinstance(parsed, WorkflowTurn): + raise TypeError("parse_input must return WorkflowTurn.") + return parsed + + async def _handle_invoke(self, request: Request) -> Response: + """Invoke the native workflow with a trusted request.""" + context = get_request_context() + try: + hosted_scope = invocation_request_scope(self.config, request, context) + partition_key = invocation_partition_key(self.config, context=context, scope=hosted_scope) + except RuntimeError as exc: + logger.error("Failed to resolve Invocations session: %s", exc) + return JSONResponse({"error": str(exc)}, status_code=500) + + try: + turn = await self._parse(request) + except (TypeError, ValueError) as exc: + return JSONResponse({"error": str(exc)}, status_code=400) + except Exception: + logger.exception("Failed to prepare Invocations request") + return JSONResponse({"error": "Failed to prepare invocation request."}, status_code=500) + + scope = hosted_scope or FoundryRequestScope.from_context(self.config, context) + if turn.stream: + + async def stream_response() -> AsyncGenerator[str]: + token: Token[FoundryAgentRequestContext] | None = set_request_context(context) + events = self._workflow_events(request, turn, scope, context, partition_key) + try: + async for frame in events: + wire_frame = _sse(frame["type"], frame) + reset_request_context(token) + token = None + yield wire_frame + token = set_request_context(context) + reset_request_context(token) + token = None + yield _sse("done", {"session_id": scope.session_id}) + except Exception as exc: + logger.exception("Invocations workflow stream failed") + message, status, code = _workflow_failure(exc) + if token is not None: + reset_request_context(token) + token = None + yield _sse("error", {"message": message, "code": code, "status": status}) + finally: + if token is None: + token = set_request_context(context) + try: + with CancelScope(shield=True): + await events.aclose() + finally: + reset_request_context(token) + + return _WorkflowStreamingResponse(stream_response(), keep_alive=self.config.sse_keepalive_interval > 0) + + try: + frames = [frame async for frame in self._workflow_events(request, turn, scope, context, partition_key)] + return JSONResponse({"output": frames}) + except Exception as exc: + logger.exception("Invocations workflow request failed") + message, status, _ = _workflow_failure(exc) + return JSONResponse({"error": message}, status_code=status) + + async def _workflow_events( + self, + request: Request, + turn: WorkflowTurn[Any], + scope: FoundryRequestScope, + context: FoundryAgentRequestContext, + partition_key: str | tuple[str, str], + ) -> AsyncGenerator[dict[str, Any]]: + frames: list[dict[str, Any]] = [] + snapshot_bytes = 0 + deferred: list[dict[str, Any]] = [] + defer = False + lock = self._session_locks.setdefault(partition_key, asyncio.Lock()) + async with lock: + workflow = await self._workflow_resolver.resolve(request) + run = await HostedWorkflowRun.prepare( + workflow, + scope=scope, + response_id=context.call_id or uuid.uuid4().hex, + config=self.config, + platform_context=context, + checkpoint_store_provider=self._checkpoint_storage_provider, + lineage_id="invocations", + stored=True, + replay_completed=True, + fresh_factory=True, + reclaim_superseded=True, + ) + if run.has_completed_output: + for frame in _workflow_snapshot_frames(run.snapshot): + yield frame + return + try: + turn = run.validate_turn(turn) + except (TypeError, ValueError) as exc: + raise _WorkflowRequestError(str(exc)) from exc + if ( + run.stored + and turn.input is not None + and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) + ): + try: + self._checkpoint_storage_provider.validate_checkpoint_value(turn.input) + except WorkflowCheckpointException as exc: + raise _WorkflowRequestError( + "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " + 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' + ) from exc + if ( + run.stored + and turn.responses is not None + and isinstance(self._checkpoint_storage_provider, CheckpointStoreProvider) + ): + try: + for reply in turn.responses.values(): + self._checkpoint_storage_provider.validate_checkpoint_value(reply) + except WorkflowCheckpointException as exc: + raise _WorkflowRequestError( + "Stored native workflow reply uses an application type that is not checkpoint-allowlisted. " + 'Configure CheckpointStoreProvider(allowed_checkpoint_types=["module:qualname"]).' + ) from exc + committed = False + resources = AsyncExitStack() + try: + await run.claim() + try: + entered: set[int] = set() + for agent in workflow_agents(workflow): + if id(agent) not in entered and isinstance(agent, AbstractAsyncContextManager): + entered.add(id(agent)) + await resources.enter_async_context(agent) + events = run.events(turn) + resources.push_async_callback(events.aclose) + async for event in events: + frame = _workflow_frame(event) + if frame is None: + continue + frame_size = _workflow_frame_size(frame) + if ( + len(frames) >= _MAX_WORKFLOW_SNAPSHOT_EVENTS + or snapshot_bytes + frame_size > _MAX_WORKFLOW_SNAPSHOT_BYTES + ): + raise _WorkflowOutputError( + "Workflow output exceeds the bounded Invocations snapshot limit." + ) + snapshot_bytes += frame_size + frames.append(frame) + defer = defer or event.type == "request_info" + if defer: + deferred.append(frame) + else: + yield frame + finally: + # Starlette cancellation must not interrupt core/resource cleanup. + with CancelScope(shield=True): + await resources.__aexit__(*sys.exc_info()) + await run.commit({"output": frames}) + committed = True + finally: + if not committed: + failure = sys.exc_info()[1] + try: + with CancelScope(shield=True): + await run.abort() + except Exception as exc: + logger.exception("Failed to persist interrupted Invocations workflow state") + if failure is None: + raise + if isinstance(failure, Exception): + raise RuntimeError( + "Workflow invocation failed and its interrupted state could not be persisted." + ) from exc + # Pending reply authority is durable, and a reply need not wait on this stream's lock. + for frame in deferred: + yield frame diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_request.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_request.py deleted file mode 100644 index 335658f276f..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_request.py +++ /dev/null @@ -1,260 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -"""Foundry request models and per-turn options for protocol hosts.""" - -from __future__ import annotations - -import inspect -from collections.abc import Awaitable, Callable, Mapping, Sequence -from copy import deepcopy -from dataclasses import dataclass, field -from types import MappingProxyType -from typing import Any, Generic, Literal, TypeAlias, cast - -from agent_framework import AgentRunInputs, Content, Message, WorkflowInvocationKwargs -from azure.ai.agentserver.responses import ResponseContext -from azure.ai.agentserver.responses.models import CreateResponse, Item -from typing_extensions import TypeVar - -from ._scope import FoundryRequestScope - -UnsupportedOptions: TypeAlias = Literal["ignore", "warn", "error"] -InputT = TypeVar("InputT", default=Any) - - -@dataclass(frozen=True) -class WorkflowTurn(Generic[InputT]): - """A typed workflow start input or a complete batch of pending replies. - - Args: - input: Application input accepted by the start executor. ``None`` is not a start input. - responses: Replies keyed by the exact pending workflow request IDs. The host validates - the batch against its scoped checkpoint before consuming any reply authority. - client_kwargs: Existing workflow kwargs forwarded to agent clients, not raw workflow run arguments. - function_invocation_kwargs: Existing workflow kwargs forwarded to tools. Trusted hosting - identity is supplied separately and cannot be overridden. - stream: Application streaming intent for protocols with a custom request parser. - Responses continues to use its native request's ``stream`` flag. - """ - - input: InputT | None = None - responses: Mapping[str, Any] | None = None - client_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None - function_invocation_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None - stream: bool = False - - def __post_init__(self) -> None: - if (self.input is None and self.responses is None) or (self.input is not None and self.responses is not None): - raise ValueError("WorkflowTurn requires exactly one of input or non-empty responses.") - if self.responses is not None: - if ( - not isinstance(self.responses, Mapping) - or not self.responses - or any(not isinstance(key, str) or not key for key in self.responses) - ): - raise ValueError("WorkflowTurn.responses must be a non-empty mapping of request IDs to replies.") - object.__setattr__(self, "responses", MappingProxyType(dict(self.responses))) - for name in ("client_kwargs", "function_invocation_kwargs"): - value = getattr(self, name) - if value is not None and not isinstance(value, (Mapping, WorkflowInvocationKwargs)): - raise TypeError(f"WorkflowTurn.{name} must be a mapping or WorkflowInvocationKwargs.") - if not isinstance(self.stream, bool): - raise TypeError("WorkflowTurn.stream must be a boolean.") - - -@dataclass(frozen=True) -class InvocationRun: - """Messages, per-turn options, and streaming intent parsed from an Invocations request. - - Args: - messages: MAF message input for this turn, including typed `Message` or `Content` values. - options: Caller generation options for this turn, separate from agent defaults. - stream: Whether to stream framed SSE events rather than return a JSON response. - - Raises: - TypeError: If the messages, options, or streaming intent have invalid types. - """ - - messages: AgentRunInputs - options: Mapping[str, Any] = field(default_factory=lambda: dict[str, Any]()) - stream: bool = False - - def __post_init__(self) -> None: - if not ( - isinstance(self.messages, (str, Content, Message)) - or ( - isinstance(self.messages, Sequence) - and all(isinstance(message, (str, Content, Message)) for message in self.messages) - ) - ): - raise TypeError("InvocationRun.messages must be a string, Content, Message, or a sequence of them.") - if not isinstance(self.options, Mapping) or any(not isinstance(key, str) for key in self.options): - raise TypeError("InvocationRun.options must be a mapping with string keys.") - if not isinstance(self.stream, bool): - raise TypeError("InvocationRun.stream must be a boolean.") - - -_HOST_CONTROLLED_FIELDS = frozenset({ - "agent", - "agent_reference", - "agent_session_id", - "background", - "call_id", - "continuation_token", - "conversation", - "conversation_id", - "extra_body", - "input", - "previous_response_id", - "response_id", - "service_session_id", - "session_id", - "store", - "stream", - "user", - "user_id", -}) -_OPTION_NAMES = {"max_output_tokens": "max_tokens", "parallel_tool_calls": "allow_multiple_tool_calls"} -_NATIVE_FIELDS = frozenset(CreateResponse.__annotations__) - - -def response_run_options(request: CreateResponse) -> dict[str, Any]: - """Translate native generation fields, with flattened extra-body values winning collisions.""" - native: dict[str, Any] = {} - extra: dict[str, Any] = {} - nested_extra: dict[str, Any] = {} - for name, value in request.items(): - if value is None or (name == "model" and value == ""): - continue - if name == "extra_body": - if not isinstance(value, Mapping) or any( - not isinstance(key, str) for key in cast(Mapping[object, object], value) - ): - raise TypeError("extra_body must be a mapping of model options.") - if "extra_body" in value: - raise ValueError("Nested extra_body is not supported; flatten provider options instead.") - nested_extra.update(cast(Mapping[str, Any], value)) - elif name in _HOST_CONTROLLED_FIELDS: - continue - elif name in _NATIVE_FIELDS: - native[_OPTION_NAMES.get(name, name)] = value - else: - extra[name] = value - return { - **native, - **{name: value for name, value in extra.items() if name not in _HOST_CONTROLLED_FIELDS}, - **{name: value for name, value in nested_extra.items() if name not in _HOST_CONTROLLED_FIELDS}, - } - - -class HostedResponseRequest: - """Expose a copied request, trusted scope, and only this turn's input to an options hook.""" - - def __init__( - self, - request: CreateResponse, - context: ResponseContext, - scope: FoundryRequestScope, - options: Mapping[str, Any], - ) -> None: - self.request: Mapping[str, Any] = MappingProxyType(deepcopy(dict(request))) - self.scope = scope - self.response_id = context.response_id - self.conversation_id = context.conversation_id - self._context = context - self._options: Mapping[str, Any] = MappingProxyType(dict(options)) - self._workflow_responses: Callable[[], Awaitable[dict[str, Any]]] | None = None - - @property - def options(self) -> Mapping[str, Any]: - """Return this turn's effective caller options after the hook.""" - return self._options - - def set_options(self, options: Mapping[str, Any]) -> None: - """Replace the caller options without changing the agent's defaults.""" - self._options = MappingProxyType(dict(options)) - - async def get_input_items(self) -> list[Item]: - """Read only this turn's input items, not earlier conversation history.""" - return list(await self._context.get_input_items()) - - async def get_input_text(self) -> str | None: - """Read this turn's text input, if present.""" - return await self._context.get_input_text() - - async def get_workflow_responses(self) -> dict[str, Any]: - """Decode replies against this turn's exact scoped workflow checkpoint. - - Use this explicitly in ``parse_response`` to resume a native workflow. - Unknown, duplicate, stale, and incomplete reply batches are rejected before - the host claims the workflow or executes any response handler. - """ - if self._workflow_responses is None: - raise RuntimeError("Workflow replies require a native workflow host and a pending stored checkpoint.") - return await self._workflow_responses() - - def _set_workflow_responses(self, loader: Callable[[], Awaitable[dict[str, Any]]]) -> None: - self._workflow_responses = loader - - -OptionsHook: TypeAlias = Callable[ - [HostedResponseRequest, dict[str, Any]], - Mapping[str, Any] | Awaitable[Mapping[str, Any]], -] - - -async def prepare_response_options(request: HostedResponseRequest, hook: OptionsHook | None) -> None: - """Apply a synchronous or asynchronous developer hook to a copy of caller options.""" - if hook is None: - return - result = hook(request, dict(request.options)) - if inspect.isawaitable(result): - result = await result - if not isinstance(result, Mapping): - raise TypeError("prepare_options must return a mapping of MAF run options.") - request.set_options(result) - - -async def response_input_messages(request: HostedResponseRequest) -> list[Message]: - """Convert only this Responses turn's input items to the legacy workflow message contract. - - This explicit migration helper does not load outer history, restore checkpoints, - or decode pending native workflow replies. Native workflows should prefer typed - application inputs and use ``request.get_workflow_responses()`` for resumable pauses. - """ - from ._responses import _items_to_messages # pyright: ignore[reportPrivateUsage] - - return await _items_to_messages(await request.get_input_items(), approval_storage=None) - - -def validate_request_options(options: Mapping[str, Any]) -> None: - """Keep hosting identity, storage decisions, and private continuation out of model options.""" - reserved = _HOST_CONTROLLED_FIELDS.intersection(options) - if reserved: - raise ValueError(f"prepare_options cannot set host-controlled fields: {', '.join(sorted(reserved))}.") - - -def validate_default_transport_options(defaults: Mapping[str, Any], *, allow_agent_store: bool) -> None: - """Reject transport overrides that would bypass the host's inner storage and identity decisions.""" - extra_body = defaults.get("extra_body") - if extra_body is None: - return - if not isinstance(extra_body, Mapping) or any( - not isinstance(key, str) for key in cast(Mapping[object, object], extra_body) - ): - raise TypeError("Agent default extra_body must be a mapping of model options.") - reserved = _HOST_CONTROLLED_FIELDS.intersection(cast(Mapping[str, Any], extra_body)) - if allow_agent_store: - reserved -= {"store"} - if reserved: - raise ValueError( - "Agent default extra_body cannot set host-controlled fields: " - f"{', '.join(sorted(reserved))}. Use explicit agent defaults or history_source='service'." - ) - - -def validate_unsupported_options(mode: str) -> UnsupportedOptions: - """Reject misspelled unsupported-options policies at host construction.""" - if mode not in ("ignore", "warn", "error"): - raise ValueError("unsupported_options must be 'ignore', 'warn', or 'error'.") - return mode diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py deleted file mode 100644 index 1ee036dd11e..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py +++ /dev/null @@ -1,4086 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from __future__ import annotations - -import asyncio -import base64 -import ipaddress -import json -import logging -import os -import re -import uuid -import warnings -from collections.abc import ( - AsyncGenerator, - AsyncIterable, - AsyncIterator, - Awaitable, - Callable, - Generator, - Mapping, - Sequence, -) -from contextlib import AbstractAsyncContextManager, AsyncExitStack, aclosing, suppress -from copy import copy -from dataclasses import asdict, dataclass, is_dataclass -from typing import TYPE_CHECKING, Any, Generic, Literal, TypeGuard, TypeVar, cast -from urllib.parse import urlparse - -from agent_framework import ( - AgentResponse, - AgentResponseUpdate, - AgentSession, - ChatOptions, - CheckpointStorage, - ComputerSafetyCheck, - Content, - ContextProvider, - FinishReason, - HistoryProvider, - InMemoryHistoryProvider, - Message, - RawAgent, - ResponseStream, - SessionStore, - SupportsAgentRun, - UsageDetails, - Workflow, - WorkflowAgent, - add_usage_details, - get_namespaced_value, -) -from agent_framework import ( - Annotation as FrameworkAnnotation, -) -from agent_framework._mcp import _MCP_TOOL_RESULT_HOST_PAYLOAD_KEY # pyright: ignore[reportPrivateUsage] -from agent_framework._telemetry import mark_feature_used -from agent_framework._types import _get_annotation_text_span_regions # pyright: ignore[reportPrivateUsage] -from agent_framework._workflows._agent import _StreamingRoleGate # pyright: ignore[reportPrivateUsage] -from agent_framework.exceptions import AgentFrameworkException -from agent_framework.observability import OBSERVABILITY_SETTINGS -from azure.ai.agentserver.core import get_request_context -from azure.ai.agentserver.responses import ( - ResponseContext, - ResponseProviderProtocol, - ResponsesServerOptions, -) -from azure.ai.agentserver.responses._id_generator import IdGenerator -from azure.ai.agentserver.responses.aio import ResponseEventStream -from azure.ai.agentserver.responses.hosting import ResponsesAgentServerHost -from azure.ai.agentserver.responses.models import ( - Annotation as ResponsesAnnotation, -) -from azure.ai.agentserver.responses.models import ( - ComputerAction, - ComputerCallSafetyCheckParam, - ComputerScreenshotImage, - ContainerFileCitationBody, - CreateResponse, - FileCitationBody, - FilePath, - FunctionShellAction, - FunctionShellCallOutputContent, - FunctionShellCallOutputExitOutcome, - FunctionShellCallOutputTimeoutOutcome, - Item, - ItemReasoningItem, - LocalEnvironmentResource, - MessageContent, - OAuthConsentRequestOutputItem, - OutputItem, - OutputItemComputerToolCall, - OutputItemComputerToolCallOutput, - OutputItemReasoningItem, - OutputMessageContent, - ResponseIncompleteReason, - ResponseStreamEvent, - ResponseUsage, - ResponseUsageInputTokensDetails, - ResponseUsageOutputTokensDetails, - UrlCitationBody, -) -from azure.ai.agentserver.responses.streaming._builders import ( - OutputItemBuilder, - OutputItemFunctionCallBuilder, - OutputItemMcpCallBuilder, - OutputItemMessageBuilder, - ReasoningSummaryPartBuilder, - RefusalContentBuilder, - TextContentBuilder, -) -from azure.ai.agentserver.responses.streaming._checkpoint import ResponseCheckpointEvent -from mcp import MCPError - -from ._agent_source import is_agent, resolve_agent, validate_agent_source -from ._feature_usage import FeatureIndex -from ._request import ( - HostedResponseRequest, - OptionsHook, - UnsupportedOptions, - WorkflowTurn, - prepare_response_options, - response_run_options, - validate_default_transport_options, - validate_request_options, - validate_unsupported_options, -) -from ._scope import FoundryRequestScope -from ._state_store import ( - AgentSessionStoreProvider, - CheckpointStoreProvider, - ContextScopedStoreProvider, - FunctionApprovalStore, - FunctionApprovalStoreProvider, - StoreProvider, -) -from ._workflow_source import WorkflowSource, validate_workflow_source - -if TYPE_CHECKING: - from ._workflow_responses import NativeResponsesWorkflow - -logger = logging.getLogger(__name__) - -_MODEL_OUTPUT_KIND_KEY = "model_output_kind" -_MODEL_OUTPUT_REFUSAL = "refusal" -_HOSTED_RESPONSES_HISTORY_SOURCE_ID = "_foundry_responses_history" -_HOSTED_PROVIDER_STATE_KEY = "_foundry_provider_background" -_HOSTED_SOURCE_CONVERSATION_KEY = "_foundry_source_conversation" -_HOSTED_SERVICE_CHILD_KEY = "_foundry_service_child" -_HOSTED_CONVERSATION_CLAIM_KEY = "_foundry_conversation_claim" -_HOSTED_CONVERSATION_COMMITTED_KEY = "_foundry_conversation_committed" -_HOSTED_PROVIDER_OUTPUT_COUNT_KEY = "_foundry_provider_output_count" -_HOSTED_PROVIDER_USAGE_KEY = "_foundry_provider_usage" -_PENDING_CONTAINER_FILE_CITATIONS_KEY = "_foundry_pending_container_file_citations" -_CONTAINER_FILE_CITATIONS_KEY = "container_file_citations" -_PUBLIC_STREAM_FAILURE_MESSAGE = "An internal server error occurred." - -_HistorySource = Literal["agent_server", "agent", "service"] -_AGENT_SOURCE_UNSET = object() - - -class _PublicResponseFailure(Exception): - """Mark a vetted failure message as safe for client-visible serialization.""" - - def __init__(self, error: Exception, public_message: str | None = None) -> None: - super().__init__(public_message or str(error) or type(error).__name__) - self.error = error - self.public_message = public_message or str(error) or type(error).__name__ - - -async def _set_agent_session(store: SessionStore, session_id: str, session: AgentSession) -> None: - """Persist a session while preserving the established public conflict contract.""" - try: - await store.set(session_id, session) - except RuntimeError as exc: - if str(exc) == "Another request advanced this agent session; reload before writing.": - raise _PublicResponseFailure(exc) from None - raise - - -def _log_response_failure(phase: str, response_id: str, exc: BaseException) -> None: - """Log a correlated hosted response failure under the sensitive-data policy.""" - diagnostic_exception = exc.error if isinstance(exc, _PublicResponseFailure) else exc - if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED: - logger.error( - "Hosted Responses failure (phase=%s, response_id=%s, error_type=%s).", - phase, - response_id, - type(diagnostic_exception).__name__, - exc_info=( - type(diagnostic_exception), - diagnostic_exception, - diagnostic_exception.__traceback__, - ), - ) - else: - logger.error( - "Hosted Responses failure (phase=%s, response_id=%s, error_type=%s).", - phase, - response_id, - type(diagnostic_exception).__name__, - ) - - -@dataclass(frozen=True) -class _ContainerFileCitation: - container_id: str - file_id: str - filename: str - - -def _has_container_filename_boundaries(text: str, start_index: int, end_index: int) -> bool: - """Return whether a filename match is not embedded in another filename-like token.""" - before = text[start_index - 1] if start_index > 0 else None - after = text[end_index] if end_index < len(text) else None - return (before is None or not (before.isalnum() or before in "._-")) and ( - after is None or not (after.isalnum() or after in "._-") - ) - - -def _container_file_citations_from_function_result(content: Content) -> list[_ContainerFileCitation]: - """Extract validated container file citations from a core-preserved MCP Host payload.""" - raw_payload = content.additional_properties.get(_MCP_TOOL_RESULT_HOST_PAYLOAD_KEY) - if not isinstance(raw_payload, Mapping): - return [] - payload = cast(Mapping[str, Any], raw_payload) - - metadata_blocks: list[Mapping[str, Any]] = [] - raw_result_meta = payload.get("_meta") - if isinstance(raw_result_meta, Mapping): - metadata_blocks.append(cast(Mapping[str, Any], raw_result_meta)) - - raw_content = payload.get("content") - if isinstance(raw_content, Sequence) and not isinstance(raw_content, (str, bytes, bytearray)): - for raw_item in cast(Sequence[Any], raw_content): - if not isinstance(raw_item, Mapping): - continue - item = cast(Mapping[str, Any], raw_item) - raw_item_meta = item.get("_meta") - if isinstance(raw_item_meta, Mapping): - metadata_blocks.append(cast(Mapping[str, Any], raw_item_meta)) - - citations: list[_ContainerFileCitation] = [] - for metadata in metadata_blocks: - raw_citations = metadata.get(_CONTAINER_FILE_CITATIONS_KEY) - if isinstance(raw_citations, str): - try: - raw_citations = json.loads(raw_citations) - except (json.JSONDecodeError, RecursionError): - continue - - citation_values: list[Any] - if isinstance(raw_citations, Mapping): - citation_values = [cast(Mapping[str, Any], raw_citations)] - elif isinstance(raw_citations, Sequence) and not isinstance(raw_citations, (str, bytes, bytearray)): - citation_values = list(cast(Sequence[Any], raw_citations)) - else: - continue - - fallback_container_id = metadata.get("container_id") - for raw_citation in citation_values: - if not isinstance(raw_citation, Mapping): - continue - citation = cast(Mapping[str, Any], raw_citation) - container_id = citation.get("container_id") or fallback_container_id - file_id = citation.get("file_id") - filename = citation.get("filename") - if not isinstance(container_id, str) or not container_id: - continue - if not isinstance(file_id, str) or not file_id: - continue - if not isinstance(filename, str) or not filename: - continue - citations.append( - _ContainerFileCitation( - container_id=container_id, - file_id=file_id, - filename=filename, - ) - ) - - return citations - - -def _annotations_to_output_text( - annotations: Sequence[FrameworkAnnotation] | None, -) -> list[ResponsesAnnotation]: - """Convert framework citations into Responses output-text annotations. - - Source-coordinate regions describe the cited source, not generated content, and - are not emitted as output-text spans. - """ - if not annotations or not isinstance(annotations, Sequence) or isinstance(annotations, (str, bytes)): - return [] - - output_annotations: list[ResponsesAnnotation] = [] - for annotation in annotations: - if not isinstance(annotation, Mapping): - continue - annotation_mapping = cast("Mapping[str, Any]", annotation) - if annotation_mapping.get("type") != "citation": - continue - - raw_properties = annotation_mapping.get("additional_properties") - properties: Mapping[str, Any] = ( - cast("Mapping[str, Any]", raw_properties) if isinstance(raw_properties, Mapping) else {} - ) - regions = _get_annotation_text_span_regions(annotation_mapping) - file_id = annotation_mapping.get("file_id") - url = annotation_mapping.get("url") - title = annotation_mapping.get("title") - container_id = properties.get("container_id") - index = properties.get("index") - - if isinstance(container_id, str) and container_id and isinstance(file_id, str) and file_id: - for region in regions: - output_annotations.append( - ContainerFileCitationBody( - type="container_file_citation", - container_id=container_id, - file_id=file_id, - filename=url if isinstance(url, str) else "", - start_index=region["start_index"], - end_index=region["end_index"], - ) - ) - elif isinstance(url, str) and url and not file_id: - for region in regions: - output_annotations.append( - UrlCitationBody( - type="url_citation", - url=url, - title=title if isinstance(title, str) else "", - start_index=region["start_index"], - end_index=region["end_index"], - ) - ) - elif ( - isinstance(file_id, str) and file_id and isinstance(url, str) and url and type(index) is int and index >= 0 - ): - output_annotations.append( - FileCitationBody( - type="file_citation", - file_id=file_id, - filename=url, - index=index, - ) - ) - elif isinstance(file_id, str) and file_id and type(index) is int and index >= 0: - output_annotations.append( - FilePath( - type="file_path", - file_id=file_id, - index=index, - ) - ) - - return output_annotations - - -def _is_refusal_text_content(content: Content) -> bool: - return ( - content.type == "text" - and get_namespaced_value( - content.additional_properties, - "core", - _MODEL_OUTPUT_KIND_KEY, - content.additional_properties.get(_MODEL_OUTPUT_KIND_KEY), - ) - == _MODEL_OUTPUT_REFUSAL - ) - - -def _refusal_additional_properties() -> dict[str, Any]: - return { - "core": {_MODEL_OUTPUT_KIND_KEY: _MODEL_OUTPUT_REFUSAL}, - _MODEL_OUTPUT_KIND_KEY: _MODEL_OUTPUT_REFUSAL, - } - - -def _openai_value(content: Content, key: str, *legacy_keys: str, default: Any = None) -> Any: - value = get_namespaced_value(content.additional_properties, "openai", key, None) - if value is not None: - return value - for legacy_key in legacy_keys: - if legacy_key in content.additional_properties: - return content.additional_properties[legacy_key] - return default - - -def _validate_checkpoint_context_id(context_id: str) -> None: - """Validate that a checkpoint context ID is a single safe path component in case file-based storage is used.""" - if ( - not context_id - or "/" in context_id - or "\\" in context_id - or "\x00" in context_id - or context_id.strip(".") == "" - or os.path.isabs(context_id) - or os.path.splitdrive(context_id)[0] - ): - raise _PublicResponseFailure(RuntimeError(f"Invalid context id: {context_id!r}")) - - -def _is_hosted_responses_history_sentinel(provider: ContextProvider) -> bool: - """Return whether ``provider`` is the host's transient history buffer.""" - return ( - isinstance(provider, InMemoryHistoryProvider) - and provider.source_id == _HOSTED_RESPONSES_HISTORY_SOURCE_ID - and provider.load_messages - and provider.store_inputs - and not provider.store_context_messages - and provider.store_outputs - ) - - -def _reject_busy_conversation(session: AgentSession) -> None: - if session.state.get(_HOSTED_CONVERSATION_CLAIM_KEY) is not None: - raise _PublicResponseFailure( - RuntimeError( - "A service-backed conversation already has an in-flight turn. Wait for it to finish; " - "if it failed or was cancelled after dispatch, start a new conversation rather than " - "reusing a possibly changed provider thread." - ) - ) - - -def _request_without_reserved_metadata(request: CreateResponse) -> CreateResponse: - """Copy a create request with any client-supplied reserved metadata bag removed. - - ``ResponseEventStream`` seeds the response from the request, and host recovery later trusts - that bag for incomplete status, provider cursors, usage, and citations. Only a persisted - response is a trusted source. The caller's request mapping is left unchanged. - """ - raw_metadata = request.get("metadata") - if not isinstance(raw_metadata, Mapping): - return request - # Metadata is a TypedDict of known public keys. Cast so a client-supplied reserved key is visible. - metadata = cast(Mapping[str, Any], raw_metadata) - if _RESERVED_RESPONSE_METADATA_KEY not in metadata: - return request - public_metadata: dict[str, Any] = { - key: value for key, value in metadata.items() if key != _RESERVED_RESPONSE_METADATA_KEY - } - seeded: dict[str, Any] = dict(request) - if public_metadata: - seeded["metadata"] = public_metadata - else: - seeded.pop("metadata", None) - return cast(CreateResponse, seeded) - - -def _create_response_event_stream(context: ResponseContext, request: CreateResponse) -> ResponseEventStream: - """Create a response stream from recovery state or the originating request.""" - if context.is_recovery: - persisted_response = context.persisted_response - if persisted_response is not None: - return ResponseEventStream(response=persisted_response, response_id=context.response_id) - return ResponseEventStream( - request=_request_without_reserved_metadata(request), - response_id=context.response_id, - ) - - -def _hosted_internal_metadata(stream: ResponseEventStream) -> dict[str, Any]: - """Keep private state in the durable envelope, outside the public metadata quota. - - The SDK strips its reserved ``internal_metadata`` key at every wire boundary. Older - checkpoints used its quota-limited response metadata bag; copy that trusted state only - when the envelope has no private bag of its own. - """ - metadata = stream.response.get("internal_metadata") - if metadata is None: - metadata = dict(stream.internal_metadata) - stream.response["internal_metadata"] = metadata - if not isinstance(metadata, dict): - raise RuntimeError("The persisted host metadata is invalid.") - return cast(dict[str, Any], metadata) - - -def _read_persisted_incomplete_reason(stream: ResponseEventStream) -> str | None: - """Return a truncating finish reason restored from a trusted response snapshot.""" - persisted = _hosted_internal_metadata(stream).get(_INCOMPLETE_REASON_KEY) - if isinstance(persisted, str): - return persisted - details = stream.response.get("incomplete_details") - if isinstance(details, Mapping): - reason = cast(Mapping[str, Any], details).get("reason") - if isinstance(reason, str): - return reason - return None - - -def _agent_response_updates(response: AgentResponse[Any], response_id: str) -> list[AgentResponseUpdate]: - """Project a completed inner response without exposing its private provider token or ID.""" - updates = [ - AgentResponseUpdate( - contents=list(message.contents), - role=message.role, - author_name=message.author_name, - response_id=response_id, - message_id=message.message_id or uuid.uuid4().hex, - ) - for message in response.messages - ] - if response.usage_details is not None: - updates.append(AgentResponseUpdate(contents=[Content.from_usage(response.usage_details)])) - if response.finish_reason is not None: - if not updates: - updates.append(AgentResponseUpdate()) - updates[-1].finish_reason = FinishReason(response.finish_reason) - return updates - - -_T = TypeVar("_T") - - -async def _await_before_signal( - operation: Callable[[], Awaitable[_T]], *signals: asyncio.Event -) -> tuple[bool, _T | None]: - """Race a provider await against lifecycle signals without discarding a completed continuation token.""" - if any(signal.is_set() for signal in signals): - return False, None - task = asyncio.ensure_future(operation()) - waiters = [asyncio.ensure_future(signal.wait()) for signal in signals] - try: - finished, _ = await asyncio.wait([task, *waiters], return_when=asyncio.FIRST_COMPLETED) - if task in finished: - return True, await task - return False, None - finally: - if not task.done(): - task.cancel() - with suppress(asyncio.CancelledError): - await task - for waiter in waiters: - waiter.cancel() - await asyncio.gather(*waiters, return_exceptions=True) - - -# Sentinel put on the internal queue by _SignalledIterator's driver task to signal that the -# wrapped iterator is exhausted (distinct from `None`, which is a valid item value). -_STOP_SENTINEL: Any = object() - - -class _SignalledIterator(Generic[_T]): - """Wraps an async iterator, stopping early as soon as any of ``events`` fires. - - Plain ``async for update in agent.run(...): if event.is_set(): break`` only observes ``event`` - once ``run()`` actually yields an item -- if it's suspended on a single slow model or tool call - with no intermediate item, the signal is invisible until that call resolves. This drives the - wrapped iterator from a single persistent background task and races each produced item against - ``events`` via ``asyncio.wait`` instead, so a signal is observed immediately. - - The background task (``_drive``) is required (rather than spawning a fresh task per step) - because some cleanup run by the wrapped iterator (e.g. observability span teardown) resets a - contextvar token set on an earlier call and requires every call against it to share the same - async context. - - If an event and a new item becomes ready at the same time, the event takes priority and the item - is discarded. Cancelling the background task while it's mid-call interrupts a suspended model or - tool call, then the driver closes the underlying stream in ``finally``. - - Callers MUST drive this through ``contextlib.aclosing`` (or an equivalent try/finally calling - ``aclose()``): ``__anext__`` only cancels the driver task on its own signalled/exhausted paths, so - if the consumer of ``async for`` raises instead (e.g. while processing a yielded item), the driver - task -- and the real agent/workflow run it's pumping -- would otherwise be silently abandoned. - """ - - def __init__( - self, - iterator: AsyncIterator[_T], - *events: asyncio.Event, - stamp: Callable[[], Awaitable[Any]] | None = None, - ) -> None: - """Wrap an async iterator, stopping early if any of ``events`` fires. - - Args: - iterator: The async iterator to wrap. - events: One or more asyncio.Event objects to watch for. If any of them is set, iteration stops early. - stamp: Optional coroutine function the driver awaits right after the wrapped iterator produces an - item and before it is advanced again. Its result is exposed as :attr:`stamp` while that item - is the current one, which lets a consumer observe state (e.g. the latest persisted workflow - checkpoint) as it was when the item was produced rather than when it is consumed: the driver - runs one item ahead, so by consumption time the wrapped iterator may already have moved on. - """ - self._iterator = iterator - self._events = events - self._stamp_fn = stamp - self._stamp: Any = None - self._signalled = False - # The queue is used to communicate items from the background driver task to the main iteration loop. - self._queue: asyncio.Queue[Any] = asyncio.Queue(maxsize=1) - # The background task that drives the wrapped iterator. - self._driver: asyncio.Task[None] | None = None - - @property - def signalled(self) -> bool: - """Whether iteration stopped early due to an event being set. - - ``signalled`` is only set when iteration stopped early because of an event -- never on ordinary - exhaustion -- so callers can tell "the agent/workflow finished" apart from "we gave up waiting". - """ - return self._signalled - - @property - def stamp(self) -> Any: - """The ``stamp`` result taken when the current item was produced (``None`` without a ``stamp``).""" - return self._stamp - - def __aiter__(self) -> _SignalledIterator[_T]: - return self - - async def _drive(self) -> None: - """Pull items from the wrapped iterator into ``self._queue`` for the object's lifetime.""" - try: - while True: - try: - item: Any = await self._iterator.__anext__() - stamp = await self._stamp_fn() if self._stamp_fn is not None else None - except StopAsyncIteration: - await self._queue.put(_STOP_SENTINEL) - return - except Exception as exc: - await self._queue.put(exc) - return - await self._queue.put((item, stamp)) - finally: - iterator: AsyncIterator[_T] = self._iterator - if isinstance(iterator, ResponseStream): - await cast(ResponseStream[_T, Any], iterator).close() - else: - close = getattr(iterator, "aclose", None) - if close is not None: - await close() - - async def __anext__(self) -> _T: - if self._driver is None: - self._driver = asyncio.ensure_future(self._drive()) - - # Create the background tasks for monitoring the events and the queue. - waiters = [asyncio.ensure_future(event.wait()) for event in self._events] - get_task = asyncio.ensure_future(self._queue.get()) - try: - # Waits until at least one of the tasks is completed. - await asyncio.wait([get_task, *waiters], return_when=asyncio.FIRST_COMPLETED) - if any(waiter.done() for waiter in waiters): - self._signalled = True - self._driver.cancel() - with suppress(BaseException): - await self._driver - get_task.cancel() - with suppress(BaseException): - await get_task - raise StopAsyncIteration - item = get_task.result() - finally: - for waiter in waiters: - if not waiter.done(): - waiter.cancel() - for waiter in waiters: - with suppress(BaseException): - await waiter - if item is _STOP_SENTINEL: - raise StopAsyncIteration - if isinstance(item, Exception): - raise item - item, self._stamp = item - return cast(_T, item) - - async def aclose(self) -> None: - """Cancel the background driver task, if any, and wait for it to finish. - - Safe to call unconditionally: a no-op if the driver was never started, and cancelling an - already-finished task (normal exhaustion or a prior signalled stop) is also a no-op. - """ - if self._driver is None: - return - self._driver.cancel() - with suppress(BaseException): - await self._driver - - -# Reserved response metadata key pinning the workflow checkpoint that was current at the moment of -# the last successfully persisted response-stream checkpoint. Recovery MUST resume from this specific -# checkpoint if it exists, not simply the latest one in checkpoint_storage: the workflow may have -# saved further checkpoints after it but before a crash, without their output ever being durably -# recorded in response.output. If this key is missing, the workflow will resume from the latest -# checkpoint in storage (if any), or replay the original input if none exists as no output was ever -# durably persisted. -_LATEST_CHECKPOINT_ID_KEY = "_last_checkpoint_id" -# ``internal_metadata`` key carrying a truncating finish reason across resilient checkpoints, so a -# turn cut short before a crash still ends ``incomplete`` after recovery. -_INCOMPLETE_REASON_KEY = "_incomplete_reason" -# SDK-reserved response metadata entry. It round-trips server-owned recovery state (this reason, -# provider cursors, usage, and citations). Clients may send the same key; it is never trusted on a -# fresh request. Foundry counts it toward the 16-entry public metadata limit. -_RESERVED_RESPONSE_METADATA_KEY = "_internal_metadata" - - -# Foundry Toolbox Auth integration -# Consent-URL error code returned by the Foundry MCP gateway when calling `/list` -CONSENT_ERROR_CODE = -32006 - -_OAUTH_HOST_PATTERN = re.compile(r"^[A-Za-z0-9._~-]+$") - - -@dataclass -class ConsentError: - name: str - consent_url: str - - -def _is_safe_oauth_consent_link(consent_link: object) -> TypeGuard[str]: - """Return whether a consent link is an absolute HTTPS URL safe to expose as an action.""" - if not isinstance(consent_link, str) or not consent_link: - return False - if any(char.isspace() or ord(char) < 0x20 or ord(char) == 0x7F for char in consent_link): - return False - - try: - parsed = urlparse(consent_link) - hostname = parsed.hostname - _ = parsed.port - except ValueError: - return False - - if parsed.scheme.lower() != "https" or not hostname or parsed.username is not None or parsed.password is not None: - return False - - if "%" in hostname: - return False - authority = parsed.netloc - if authority.startswith("["): - closing_bracket = authority.find("]") - if closing_bracket == -1: - return False - ipv6_literal = authority[1:closing_bracket] - suffix = authority[closing_bracket + 1 :] - if suffix and (not suffix.startswith(":") or not suffix[1:].isdigit()): - return False - try: - ipaddress.IPv6Address(ipv6_literal) - except ValueError: - return False - return True - if "[" in authority or "]" in authority or ":" in hostname: - return False - return _OAUTH_HOST_PATTERN.fullmatch(hostname) is not None - - -def _normalize_oauth_consent_origin(value: str, *, require_origin_only: bool) -> str: - """Normalize an HTTPS consent URL or configured origin for exact matching.""" - if not _is_safe_oauth_consent_link(value): - raise ValueError("OAuth consent origins must be absolute HTTPS URLs without user information.") - - parsed = urlparse(value) - if require_origin_only and (parsed.path not in ("", "/") or parsed.params or parsed.query or parsed.fragment): - raise ValueError("OAuth consent origin entries must not include a path, query, parameters, or fragment.") - - hostname = parsed.hostname - if hostname is None: # pragma: no cover - _is_safe_oauth_consent_link already proved this. - raise ValueError("OAuth consent origin must include a hostname.") - normalized_host = f"[{hostname.lower()}]" if ":" in hostname else hostname.lower() - port = parsed.port - return f"https://{normalized_host}" if port in (None, 443) else f"https://{normalized_host}:{port}" - - -def _normalize_allowed_oauth_consent_origins(allowed_origins: Sequence[str] | None) -> frozenset[str] | None: - """Normalize the host-configured consent origin allowlist, or return ``None`` when none is configured.""" - if allowed_origins is None: - return None - return frozenset(_normalize_oauth_consent_origin(origin, require_origin_only=True) for origin in allowed_origins) - - -def _is_allowed_oauth_consent_link(consent_link: object, allowed_origins: frozenset[str] | None) -> TypeGuard[str]: - """Return whether the link is safe and, when an allowlist is configured, has an allowed origin. - - ``allowed_origins`` must come from ``_normalize_allowed_oauth_consent_origins``. ``None`` keeps the - safe-HTTPS check without restricting the origin; an empty set rejects every link. - """ - if not _is_safe_oauth_consent_link(consent_link): - return False - if allowed_origins is None: - return True - return _normalize_oauth_consent_origin(consent_link, require_origin_only=False) in allowed_origins - - -def consent_url_from_error(exc: BaseException) -> list[ConsentError] | None: - """Return the consent URLs when ``exc`` wraps Foundry MCP gateway consent errors. - - Args: - exc: The exception to inspect. - - Returns: - The consent URL(s) extracted from the error, or ``None`` if no consent error was found. - """ - inner_exception = next((arg for arg in exc.args if isinstance(arg, MCPError)), None) - if inner_exception is not None and inner_exception.code == CONSENT_ERROR_CODE: - # Parse the error message - # The error message is structured with the following format: - # "tools/list failed for 1 tool source(s), succeeded for 0 tool source(s) {"errors":[{"name": ..." - # where the second part is a JSON string that can be deserialized into an object with the following shape: - # ruff: disable[commented-out-code] - # { - # "errors" : [ - # { - # "name": "Name of the MCP tool that requires consent", - # "type" : "mcp" | "a2a_preview", - # "error": { - # "code": "CONSENT_REQUIRED", - # "message": consent_url, - # } - # } - # ] - # } - # ruff: enable[commented-out-code] - try: - consent_errors: list[ConsentError] = [] - error_message_start = inner_exception.error.message.find("{") - if error_message_start == -1: - logger.warning("Consent error message does not contain JSON: %s", inner_exception.error.message) - return None - consent_details_json = inner_exception.error.message[error_message_start:] - consent_details = json.loads(consent_details_json) - if "errors" not in consent_details or not isinstance(consent_details["errors"], list): - logger.warning("Consent error message JSON does not contain 'errors' list: %s", consent_details_json) - return None - for error in consent_details["errors"]: - if ( - isinstance(error, dict) - and error.get("type") in ("mcp", "a2a_preview") # type: ignore - and "error" in error - and isinstance(error["error"], dict) - and error["error"].get("code") == "CONSENT_REQUIRED" # type: ignore - and "message" in error["error"] - ): - consent_url = error["error"]["message"] # type: ignore - if isinstance(consent_url, str): - consent_errors.append(ConsentError(name=error.get("name", "Unknown"), consent_url=consent_url)) # type: ignore - else: - logger.warning("Consent URL in error message is not a valid URL: %s", consent_url) # type: ignore - if consent_errors: - return consent_errors - except json.JSONDecodeError: - logger.warning("Failed to parse consent details JSON: %s", inner_exception.error.message) - return None - - -# endregion Foundry Toolbox Auth integration - - -@dataclass(frozen=True) -class _AgentConfiguration: - workflow: bool - agent_server_history: bool - client_stores_by_default: bool - hosted_history: bool - - -def _validate_agent_configuration( - agent: SupportsAgentRun, - history_source: _HistorySource, - options: ResponsesServerOptions | None, - *, - background_source: Literal["agent_server", "provider"] = "agent_server", -) -> _AgentConfiguration: - is_workflow_agent = isinstance(agent, WorkflowAgent) - if is_workflow_agent and agent.workflow._runner_context.has_checkpointing(): # pyright: ignore[reportPrivateUsage] - raise RuntimeError( - "There should not be a checkpoint storage already present in the workflow agent. " - "The hosting infrastructure will manage checkpoints instead." - ) - - resilient_background = bool(options and options.resilient_background) - if resilient_background and not is_workflow_agent and background_source != "provider": - raise RuntimeError( - "resilient_background=True is only supported for workflow agents. " - "Crash recovery cannot be provided for non-workflow agents." - ) - if background_source == "provider" and ( - not isinstance(agent, RawAgent) or getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", None) is not True - ): - raise RuntimeError("Provider background requires a RawAgent with a storing, resumable Responses client.") - if options and options.steerable_conversations and is_workflow_agent: - raise RuntimeError( - "steerable_conversations=True is only supported for non-workflow agents. " - "Steering cannot be provided reliably for workflow agents." - ) - - if is_workflow_agent and history_source == "service": - raise ValueError("history_source='service' is only supported for regular agents.") - - if not is_workflow_agent and isinstance(agent, RawAgent): - identity_defaults = ("session_id", "agent_session_id", "user_id", "call_id", "service_session_id") - if conflicting := [name for name in identity_defaults if agent.default_options.get(name) is not None]: - raise RuntimeError(f"Model defaults cannot supply Foundry platform identity: {', '.join(conflicting)}.") - - uses_agent_server_history = history_source == "agent_server" - client_stores_by_default = False - if not is_workflow_agent and history_source != "agent": - if not isinstance(agent, RawAgent): - raise RuntimeError( - "history_source='agent_server' and 'service' require a RawAgent so hosting can enforce downstream " - "storage. Use history_source='agent' for a custom SupportsAgentRun implementation on stored requests." - ) - - stores_by_default = getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", None) - if not isinstance(stores_by_default, bool): - raise RuntimeError( - "history_source='agent_server' or 'service' requires the chat client to declare STORES_BY_DEFAULT " - "so hosting can enforce downstream storage behavior." - ) - client_stores_by_default = stores_by_default - - if history_source == "service" and not stores_by_default: - raise RuntimeError( - "history_source='service' requires a storing client that declares STORES_BY_DEFAULT=True." - ) - - service_continuation_options = [ - name - for name in ("conversation_id", "previous_response_id", "conversation", "continuation_token", "response_id") - if agent.default_options.get(name) is not None - ] - if service_continuation_options: - raise RuntimeError( - f"history_source='agent_server' or 'service' cannot use developer defaults for downstream " - f"continuation: {', '.join(service_continuation_options)}. " - "Use history_source='agent' to retain developer-controlled continuation." - ) - - if history_source in ("agent_server", "service"): - for provider in agent.context_providers: - if isinstance(provider, HistoryProvider) and provider.load_messages: - if history_source == "agent_server" and _is_hosted_responses_history_sentinel(provider): - continue - raise RuntimeError( - "The selected history_source conflicts with a load-enabled HistoryProvider. " - "Remove that provider or select history_source='agent'." - ) - if history_source == "agent_server" and not stores_by_default and agent.default_options.get("store") is True: - raise RuntimeError( - "The chat client does not store by default, but the agent sets a downstream store option. " - "Remove that developer-owned default rather than letting hosting change it." - ) - - return _AgentConfiguration( - workflow=is_workflow_agent, - agent_server_history=uses_agent_server_history, - client_stores_by_default=client_stores_by_default, - hosted_history=uses_agent_server_history and not is_workflow_agent, - ) - - -def _initialize_agent_history(agent: SupportsAgentRun, configuration: _AgentConfiguration) -> None: - if not configuration.hosted_history or not isinstance(agent, RawAgent): - return - if not any( - _is_hosted_responses_history_sentinel(provider) - for provider in cast(Sequence[ContextProvider], agent.context_providers) - ): - agent.context_providers.append(InMemoryHistoryProvider(source_id=_HOSTED_RESPONSES_HISTORY_SOURCE_ID)) - - -# region ResponsesHostServer -class ResponsesHostServer(ResponsesAgentServerHost): - """A Responses server host for an agent or a native, typed workflow.""" - - def __init__( - self, - agent: SupportsAgentRun - | Callable[[], SupportsAgentRun | Awaitable[SupportsAgentRun]] - | object = _AGENT_SOURCE_UNSET, - *, - workflow: WorkflowSource[HostedResponseRequest] | None = None, - parse_response: Callable[[HostedResponseRequest], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]] - | None = None, - prefix: str = "", - options: ResponsesServerOptions | None = None, - store: ResponseProviderProtocol | None = None, - response_store: ResponseProviderProtocol | None = None, - agent_session_store_provider: StoreProvider[SessionStore] | None = None, - checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, - function_approval_store_provider: StoreProvider[FunctionApprovalStore] | None = None, - allowed_oauth_consent_origins: Sequence[str] | None = None, - history_source: Literal["agent_server", "agent", "service"] = "agent_server", - background_source: Literal["agent_server", "provider"] = "agent_server", - prepare_options: OptionsHook | None = None, - unsupported_options: UnsupportedOptions = "warn", - **kwargs: Any, - ) -> None: - """Initialize a ResponsesHostServer. - - Args: - agent: The agent to handle responses for, or a zero-argument sync or async callable that creates one for - each request. Use a callable for agents that keep mutable state outside `AgentSession`. - Hosting a `WorkflowAgent` here is deprecated and should be avoided: it is stateful, so one instance - must never serve requests from different users or conversations. Use `workflow=` instead. - workflow: A built, unrun native workflow for one-shot execution, or a request-aware - sync/async factory returning fresh built graphs, executors, agents, clients, tools, - and providers with stable graph/executor IDs. Cannot be combined with ``agent``. - Use a factory for continuation, pauses, or background recovery. - parse_response: Required for ``workflow``. Maps this turn's ``HostedResponseRequest`` - to a typed ``WorkflowTurn(input=...)`` or validated pending ``responses``. - Native workflows use checkpoints, not the agent's history-source policy. - prefix: The URL prefix for the server. - options: Optional server options. - store: Deprecated alias for `response_store`. - response_store: Optional response store for caller-facing persistence and history. - agent_session_store_provider: Optional provider for MAF agent session storage. - If not provided, a default `AgentSessionStoreProvider` will be used. - checkpoint_store_provider: Optional provider for workflow checkpoint storage. - If not provided, a default `CheckpointStoreProvider` will be used. - function_approval_store_provider: Optional provider for function approval storage. - If not provided, a default `FunctionApprovalStoreProvider` will be used. - allowed_oauth_consent_origins: Optional exact HTTPS origins allowed for OAuth consent links. - When omitted, hosting retains its existing safe-HTTPS validation without restricting the - destination origin. When provided, every link must match an entry; an empty sequence rejects - every link. Entries must be origins such as `"https://auth.example.com"` and must not include - a path, query, or fragment. - history_source: Who supplies prior messages to the *model*, independently of the caller's - Responses `store` flag. Defaults to `"agent_server"`: - - - `"agent_server"`: Send the prior outer Responses transcript followed by this request's input. - For a storing chat client, force its per-run `store=False` and clear any restored - `service_session_id` to avoid replaying that transcript twice. A load-enabled - `HistoryProvider` or fixed downstream continuation default conflicts with this mode. - - `"service"`: Send only this request's input. On caller `store=True`, run the inner - client with `store=True` and save its issued `service_session_id` in the host's - private MAF session store. The next stored request resumes that provider thread; - it must not branch an already-used provider conversation. - - `"agent"`: Send only this request's input and preserve the agent's developer-owned - storage choice on stored requests. With an `InMemoryHistoryProvider` and - `default_options={"store": False}`, the provider loads prior messages from - the host-persisted MAF `AgentSession`. With `default_options={"store": True}`, - the downstream service retains history instead (the previous meaning of `"agent"`). - - For example, after a stored response to "My name is Ada", a stored follow-up - "What is my name?" sends both turns to the model with `"agent_server"`, but - sends only the follow-up with `"service"` (plus a private service continuation). - With `"agent"`, the developer chooses either of those agent-owned history sources. - background_source: `"agent_server"` (default) runs background work in the Responses host - without invoking provider-native background APIs; `"provider"` opts a storing, resumable - client into provider background polling and requires `history_source="service"`. - Each poll retains the caller's run options and `background=True`; local tools must be - idempotent because a crash before the next private token is saved can replay them. - prepare_options: Developer hook to remove or replace caller model options for an agent. - unsupported_options: `"warn"` (default), `"error"`, or `"ignore"` when a custom agent - cannot accept runtime model options. - **kwargs: Additional keyword arguments. - - Note: - The *caller* controls outer persistence with `POST /responses` `store=True/False`. - `store=False` writes no host-managed session or approval state and forces supported - inner clients not to store, regardless of `history_source`. The constructor's - deprecated `store=` argument instead selects the outer response-store backend - (use `response_store=`). Outer `response.id` is the polling/continuation handle; - any inner `service_session_id` is private and never replaces it. - - 1. With `history_source="agent_server"`, - the agent must not have a load-enabled history provider: the host supplies the transcript. - 2. Context providers must not keep required state only on their Python instances, - because the hosting environment may get deactivated between requests. Provider - state carried by `AgentSession`, including `InMemoryHistoryProvider` messages in - `history_source="agent"` mode, is persisted by the configured session store. - 3. The server owns the supplied agent instance and may add hosting-specific providers. - Do not reuse the same agent with another host or invoke it directly after construction. - An agent returned by a callable belongs to that request. - 4. `resilient_background=True` supports legacy workflows and regular agents configured with - `history_source="service", background_source="provider"`. For provider background, only - a saved private continuation token can be polled after a crash; a crash before the token - is saved cannot be replayed safely. Other regular-agent runs are not crash-recoverable. - Legacy workflow background responses retain their checkpoint-based recovery behavior. - 5. Steering is temporarily unavailable for all agents. `steerable_conversations=True` fails - at construction, before starting a host or enabling the process-wide TaskManager. The current - AgentServer SDK retains futures for rejected turns after the steering queue fills. - 6. A stored, named `"service"` or `"agent"` conversation is claimed in the scoped session store - before the agent runs. Conditional writes prevent a concurrent turn from mutating the same - provider thread. If the turn fails or is cancelled, start a new conversation instead of - reusing a potentially changed downstream thread. - - Raises: - ValueError: If the history, background, or unsupported-options policy is invalid. - RuntimeError: If the agent configuration conflicts with the selected history source, - `resilient_background=True` is requested for a regular agent without provider background, or - `steerable_conversations=True` is requested while steering is unavailable. - """ - if options and options.steerable_conversations: - # TODO(foundry-hosting): Remove this guard after Azure/azure-sdk-for-python#49233 ships in an official - # azure-ai-agentserver-core wheel, the minimum and uv.lock are updated, and a concurrent - # queue-overflow regression proves rejected turns leave no pending futures. - raise RuntimeError( - "steerable_conversations=True is temporarily unavailable: the current AgentServer SDK " - "retains futures for rejected steering turns. Wait for the official fix in " - "Azure/azure-sdk-for-python#49233, then update the dependency and verify queue overflow." - ) - if history_source not in ("agent_server", "agent", "service"): - raise ValueError("history_source must be 'agent_server', 'agent', or 'service'.") - if background_source not in ("agent_server", "provider"): - raise ValueError("background_source must be 'agent_server' or 'provider'.") - if store is not None and response_store is not None: - raise ValueError("Pass response_store instead of store; they cannot be combined.") - - if background_source == "provider" and history_source != "service": - raise ValueError("Provider background requires history_source='service'.") - if background_source == "provider" and options and options.steerable_conversations: - raise ValueError("Provider background and steerable_conversations cannot be combined.") - agent_supplied = agent is not _AGENT_SOURCE_UNSET - if not agent_supplied and workflow is None: - raise ValueError("Pass exactly one of agent or workflow.") - if agent_supplied and workflow is not None: - raise ValueError("Pass exactly one of agent or workflow.") - if workflow is not None: - validate_workflow_source(workflow) - if parse_response is None or not callable(parse_response): - raise TypeError("parse_response is required for native workflow hosting.") - if history_source != "agent_server" or background_source != "agent_server": - raise ValueError( - "Native workflows use checkpoint history and AgentServer background, not agent policies." - ) - if isinstance(workflow, Workflow) and options and options.resilient_background: - raise ValueError("Native workflow background recovery requires a request-aware factory.") - else: - if parse_response is not None: - raise ValueError("parse_response is only supported with workflow.") - validate_agent_source(agent) - - resolved_agent = agent if is_agent(agent) else None - configuration = ( - _validate_agent_configuration(resolved_agent, history_source, options, background_source=background_source) - if resolved_agent is not None - else None - ) - - # No caller-owned agent state is mutated until all validation and base-host construction succeed. - self._allowed_oauth_consent_origins = _normalize_allowed_oauth_consent_origins(allowed_oauth_consent_origins) - super().__init__( - prefix=prefix, - options=options, - store=response_store if response_store is not None else store, - **kwargs, - ) - if options and options.steerable_conversations: - from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled - - set_resilient_tasks_enabled(True) - - self._agent_source = cast( - SupportsAgentRun | Callable[[], SupportsAgentRun | Awaitable[SupportsAgentRun]] | None, - None if not agent_supplied else agent, - ) - self._agent = resolved_agent - self._configuration = configuration - self._history_source: _HistorySource = history_source - self._background_source: Literal["agent_server", "provider"] = background_source - self._prepare_options = prepare_options - self._unsupported_options = validate_unsupported_options(unsupported_options) - self._host_options = options - self._uses_agent_server_history = ( - configuration.agent_server_history if configuration is not None else history_source == "agent_server" - ) - self._resilient_background = bool(options and options.resilient_background) - self._warned_workflow_agent = False - if resolved_agent is not None and configuration is not None: - _initialize_agent_history(resolved_agent, configuration) - - if store is not None: - warnings.warn("store= is deprecated; use response_store=.", DeprecationWarning, stacklevel=2) - - # Storage providers - self._checkpoint_storage_provider = ( - CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider - ) - self._session_storage_provider = ( - AgentSessionStoreProvider() if agent_session_store_provider is None else agent_session_store_provider - ) - self._function_approval_storage_provider = ( - FunctionApprovalStoreProvider() - if function_approval_store_provider is None - else function_approval_store_provider - ) - self._native_workflow: NativeResponsesWorkflow | None = None - if workflow is not None and parse_response is not None: - from ._workflow_responses import NativeResponsesWorkflow - - self._native_workflow = NativeResponsesWorkflow( - workflow, - parse_response, - config=self.config, - checkpoint_store_provider=self._checkpoint_storage_provider, - prepare_options=prepare_options, - resilient_background=self._resilient_background, - allowed_oauth_consent_origins=self._allowed_oauth_consent_origins, - ) - self._native_workflow.bind_streaming_route( - self.router, - prefix=prefix, - keep_alive=bool( - (options and options.sse_keep_alive_interval_seconds) or self.config.sse_keepalive_interval - ), - ) - if isinstance(resolved_agent, WorkflowAgent): - self._warn_legacy_workflow() - - # Lazy agent lifecycle: the agent (and any MCP tools it owns) is entered on - # the first request rather than at server startup, so that authentication - # failures during MCP connect can be surfaced to the client as an - # `oauth_consent_request` stream event instead of crashing the server. - self._agent_stack: AsyncExitStack | None = None - self._agent_init_lock = asyncio.Lock() - - self.shutdown_handler(self._cleanup_agent) - self.response_handler(self._handle_response) - - mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) - - def _warn_legacy_workflow(self) -> None: - if not self._warned_workflow_agent: - self._warned_workflow_agent = True - message = ( - "Hosting WorkflowAgent through agent= is deprecated for this beta release and should be avoided. " - "A WorkflowAgent is stateful and keeps workflow state in memory between runs, so one instance must " - "never serve requests from different users or conversations. " - "Use workflow=a_request_aware_factory with an explicit parse_response. " - "Until you migrate, pass a factory that builds a new WorkflowAgent for every request. " - "Wrapper history, context providers, approvals, and event semantics are not automatically unwrapped." - ) - warnings.warn(message, DeprecationWarning, stacklevel=3) - # Request-time calls cannot be attributed to application code, so Python's default filter hides the - # warning there; log it as well. - logger.warning("DEPRECATION: %s", message) - - async def _ensure_agent_ready(self) -> None: - """Lazily enter the agent's async context exactly once. - - On failure the partial exit stack is closed and ``_agent_stack`` is left - as ``None`` so a subsequent request (e.g. after the user completes OAuth - consent) can retry the connection. - """ - if self._agent_stack is not None: - return - async with self._agent_init_lock: - if self._agent_stack is not None: - return - agent = self._agent - if agent is None: - raise RuntimeError("A request-scoped agent cannot use the server-lifetime initialization path.") - stack = AsyncExitStack() - try: - if isinstance(agent, AbstractAsyncContextManager): - await stack.enter_async_context(cast(AbstractAsyncContextManager[Any], agent)) - except BaseException: - await stack.aclose() - raise - self._agent_stack = stack - - async def _cleanup_agent(self) -> None: - """Close the agent's async context. Registered as the server shutdown handler.""" - stack = self._agent_stack - if stack is not None: - self._agent_stack = None - await stack.aclose() - - async def _handle_response( - self, - request: CreateResponse, - context: ResponseContext, - cancellation_signal: asyncio.Event, - ) -> AsyncIterable[ResponseStreamEvent | ResponseCheckpointEvent]: - """Handle the creation of a response.""" - if self._native_workflow is not None: - async with aclosing(self._native_workflow.response_events(request, context, cancellation_signal)) as events: - async for event in events: - yield event - return - response_event_stream = _create_response_event_stream(context, request) - if context.is_steered_turn: - logger.debug("Serving steered turn (pending_input_count=%d)", context.pending_input_count) - yield response_event_stream.emit_created() - yield response_event_stream.emit_in_progress() - - terminal_event: ResponseStreamEvent | None = None - try: - try: - scope = FoundryRequestScope.from_context( - self.config, get_request_context(), local_session_id=context.response_id - ) - except (TypeError, ValueError, RuntimeError) as exc: - raise _PublicResponseFailure(exc) from None - if self._agent_source is None: - raise RuntimeError("The hosted agent source is not configured.") - agent = await resolve_agent(self._agent_source) - try: - configuration = self._configuration or _validate_agent_configuration( - agent, self._history_source, self._host_options, background_source=self._background_source - ) - except (TypeError, ValueError, RuntimeError) as exc: - raise _PublicResponseFailure(exc) from None - if self._configuration is None: - _initialize_agent_history(agent, configuration) - if configuration.workflow: - self._warn_legacy_workflow() - hosted_request: HostedResponseRequest | None = None - if configuration.workflow: - if self._prepare_options is not None: - raise _PublicResponseFailure(ValueError("prepare_options is only supported for regular agents.")) - else: - try: - hosted_request = HostedResponseRequest(request, context, scope, response_run_options(request)) - except (TypeError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - await prepare_response_options(hosted_request, self._prepare_options) - try: - validate_request_options(hosted_request.options) - except (TypeError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - except Exception as exc: - for event in self._emit_failure( - context.response_id, - "request preparation", - response_event_stream, - None, - exc, - ): - yield event - return - - async with AsyncExitStack() as resources: - inner = self._handle_prepared_response( - request, - context, - cancellation_signal, - response_event_stream, - agent, - configuration, - resources, - hosted_request, - ) - try: - async for event in inner: - if isinstance(event, Mapping) and event.get("type") in ( - "response.completed", - "response.incomplete", - "response.failed", - ): - terminal_event = event - else: - yield event - finally: - await inner.aclose() - if terminal_event is not None: - yield terminal_event - - async def _handle_prepared_response( - self, - request: CreateResponse, - context: ResponseContext, - cancellation_signal: asyncio.Event, - response_event_stream: ResponseEventStream, - agent: SupportsAgentRun, - configuration: _AgentConfiguration, - resources: AsyncExitStack, - hosted_request: HostedResponseRequest | None, - ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: - # Lazy-enter the agent (and any MCP tools it owns). The MCP client wraps gateway - # consent failures (and other connection-time errors) in AgentFrameworkException; if - # one of those is a consent error we surface the consent link to the client through - # the already-opened response stream instead of failing the request. Other exception - # types fall through to the outer handler below and become ``response.failed``. - try: - if self._configuration is not None: - await self._ensure_agent_ready() - elif isinstance(agent, AbstractAsyncContextManager): - await resources.enter_async_context(agent) - except AgentFrameworkException as ex: - consent_errors_to_emit = consent_url_from_error(ex) - if consent_errors_to_emit is None or len(consent_errors_to_emit) == 0: - for event in self._emit_failure( - context.response_id, - "agent initialization", - response_event_stream, - None, - ex, - ): - yield event - return - - invalid_consent = next( - ( - consent_error - for consent_error in consent_errors_to_emit - if not _is_allowed_oauth_consent_link( - consent_error.consent_url, self._allowed_oauth_consent_origins - ) - ), - None, - ) - if invalid_consent is not None: - validation_error = ValueError( - f"OAuth consent request for tool '{invalid_consent.name}' must include an allowed safe HTTPS " - "consent link." - ) - for event in self._emit_failure( - context.response_id, - "OAuth consent validation", - response_event_stream, - None, - _PublicResponseFailure(validation_error), - ): - yield event - return - - if request.get("store") is False: - validation_error = ValueError("OAuth consent continuation requires store=true.") - for event in self._emit_failure( - context.response_id, - "OAuth consent validation", - response_event_stream, - None, - _PublicResponseFailure(validation_error), - ): - yield event - return - - if not configuration.workflow: - try: - request_context = get_request_context() - session_storage = self._session_storage_provider.get_store( - config=self.config, platform_context=request_context - ) - previous_response_id = request.get("previous_response_id") - session_load_id = context.conversation_id or previous_response_id - session = await session_storage.get(session_load_id) if session_load_id is not None else None - if session is None: - if previous_response_id is not None and context.conversation_id is None: - raise _PublicResponseFailure( - RuntimeError( - "Cannot find an existing agent session for " - f"previous_response_id={previous_response_id}." - ) - ) - session = agent.create_session() - if context.conversation_id is not None: - _reject_busy_conversation(session) - if previous_response_id is not None and context.conversation_id is None: - if session.service_session_id is not None and session.state.get( - _HOSTED_SOURCE_CONVERSATION_KEY - ): - raise _PublicResponseFailure( - ValueError("A service-managed downstream conversation cannot be forked.") - ) - session.state.pop(_HOSTED_SOURCE_CONVERSATION_KEY, None) - if context.conversation_id is not None: - session.state[_HOSTED_SOURCE_CONVERSATION_KEY] = context.conversation_id - await _set_agent_session(session_storage, context.response_id, session) - if context.conversation_id is not None: - await _set_agent_session(session_storage, context.conversation_id, session) - except Exception as save_error: - for event in self._emit_failure( - context.response_id, - "OAuth consent session persistence", - response_event_stream, - None, - save_error, - ): - yield event - return - - for consent_error in consent_errors_to_emit: - logger.warning("Consent URL for tool '%s': %s", consent_error.name, consent_error.consent_url) - oauth_item = OAuthConsentRequestOutputItem( - id=IdGenerator.new_id("oacr"), - response_id=context.response_id, - type="oauth_consent_request", - consent_link=consent_error.consent_url, - server_label=consent_error.name, - ) - builder = response_event_stream.add_output_item(oauth_item["id"]) - yield builder.emit_added(oauth_item) - yield builder.emit_done(oauth_item) - - yield response_event_stream.emit_incomplete() - return - - tracker = _OutputItemTracker(response_event_stream, self._allowed_oauth_consent_origins) - try: - if configuration.workflow: - inner = self._handle_inner_workflow( - request, - context, - response_event_stream, - tracker, - cancellation_signal, - cast(WorkflowAgent, agent), - ) - else: - if hosted_request is None: - raise RuntimeError("A regular agent requires a prepared Responses request.") - inner = self._handle_inner_agent( - request, - context, - response_event_stream, - tracker, - cancellation_signal, - agent, - configuration, - hosted_request, - ) - - try: - async for event in inner: - yield event - except BaseException: - await inner.aclose() - raise - - if cancellation_signal.is_set() and context.client_cancelled: - # A cancelled run drains the inner generator without raising (both - # ``_handle_inner_workflow`` and ``_handle_inner_agent`` stop their - # ``_SignalledIterator`` loop and return normally once the signal fires). - # Emit nothing here so a caller cannot mistake this for a normal - # completion; the host server's cancel-aware layer synthesizes the - # cancelled terminal when the handler returns without one. Gated on - # ``client_cancelled`` (not just the signal) because steering pressure - # also sets ``cancellation_signal`` without that cause flag; a steered - # turn must still drain ``tracker.close()`` and emit its normal terminal - # below so its partial output is not misreported as a failure. - return - - for event in tracker.close(): - yield event - - if cancellation_signal.is_set() and context.client_cancelled: - # Draining ``tracker.close()`` yields events one at a time, and each - # ``yield`` above suspends this handler until the caller resumes it. - # A cancellation can arrive during that window, after the earlier check - # already passed, so it must be rechecked here, immediately before - # selecting the terminal event. Same ``client_cancelled`` gate as above. - return - - tracker.discard_pending_container_file_citations() - incomplete_reason = tracker.incomplete_reason - if tracker.oauth_consent_requested or incomplete_reason is not None: - yield response_event_stream.emit_incomplete(reason=incomplete_reason, usage=tracker.usage) - else: - yield response_event_stream.emit_completed(usage=tracker.usage) - except Exception as ex: - for event in tracker.close(): - yield event - - for event in self._emit_failure( - context.response_id, - "response production", - response_event_stream, - tracker, - ex, - ): - yield event - - async def _load_request_messages( - self, - context: ResponseContext, - *, - approval_storage: FunctionApprovalStore | None, - configuration: _AgentConfiguration | None = None, - ) -> list[Message]: - """Load the request's input and prior history concurrently, assembled for the run. - - The caller's input items and the conversation history are independent - storage round-trips with no data dependency, so they are fetched in - parallel to remove serial latency from the request critical path. The - history read is only issued when AgentServer is the history source; in - stateless single-turn requests it short-circuits without a round-trip. - - Returns the messages already ordered as model input (history precedes - input), so the message-ordering rule lives only here and callers do not - need to know the storage-result ordering. If either read fails, the - sibling task is cancelled and drained so no storage read is orphaned. - """ - uses_agent_server_history = ( - configuration.agent_server_history if configuration is not None else self._uses_agent_server_history - ) - - async def _load_input() -> list[Message]: - input_items = await context.get_input_items() - return await _items_to_messages(input_items, approval_storage=approval_storage) - - async def _load_history() -> list[Message]: - if not uses_agent_server_history: - return [] - history = await context.get_history() - return await _output_items_to_messages(history, approval_storage=approval_storage) - - input_task = asyncio.ensure_future(_load_input()) - history_task = asyncio.ensure_future(_load_history()) - try: - input_messages, history_messages = await asyncio.gather(input_task, history_task) - except BaseException: - # gather surfaces the first failure without cancelling the sibling, and a - # cancellation of this coroutine must not leave either read running. Cancel - # both and await them so no storage operation is orphaned after we unwind. - input_task.cancel() - history_task.cancel() - await asyncio.gather(input_task, history_task, return_exceptions=True) - raise - return [*history_messages, *input_messages] - - async def _handle_inner_agent( - self, - request: CreateResponse, - context: ResponseContext, - response_event_stream: ResponseEventStream, - tracker: _OutputItemTracker, - cancellation_signal: asyncio.Event, - agent: SupportsAgentRun, - configuration: _AgentConfiguration, - hosted_request: HostedResponseRequest, - ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: - """Handle a regular (non-workflow) agent. - - The response stream, tracker, and opening lifecycle events are produced - by :meth:`_handle_response`, which also converts any raised exception - into a terminal ``response.failed`` event (draining the tracker so the - SSE stream stays well-formed). - """ - provider_background = self._background_source == "provider" and request.get("background") is True - if context.is_recovery and not provider_background: - raise RuntimeError("A non-resumable agent cannot be replayed after a process crash.") - - stored = request.get("store") is not False - - request_messages_task: asyncio.Task[list[Message]] | None = None - conversation_already_committed = False - try: - if isinstance(agent, RawAgent): - try: - validate_default_transport_options( - agent.default_options, - allow_agent_store=self._history_source == "agent" and stored, - ) - except (TypeError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - if not stored: - if not isinstance(agent, RawAgent): - raise _PublicResponseFailure( - RuntimeError( - "store=false cannot guarantee that a custom agent will disable inner storage; " - "use a RawAgent or store=true." - ) - ) - continuation_defaults = [ - name - for name in ( - "conversation_id", - "previous_response_id", - "conversation", - "continuation_token", - "response_id", - ) - if agent.default_options.get(name) is not None - ] - if continuation_defaults: - raise RuntimeError( - "store=false cannot use developer defaults for downstream service continuation: " - + ", ".join(continuation_defaults) - ) - if any( - isinstance(provider, HistoryProvider) - and not isinstance(provider, InMemoryHistoryProvider) - and (provider.store_inputs or provider.store_context_messages or provider.store_outputs) - for provider in agent.context_providers - ): - raise _PublicResponseFailure( - RuntimeError( - "store=false cannot prevent an external HistoryProvider from persisting responses. " - "Use an InMemoryHistoryProvider or store=true." - ) - ) - if self._history_source == "agent": - stores_by_default = getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", None) - if not isinstance(stores_by_default, bool): - raise RuntimeError( - "store=false with history_source='agent' requires a client declaring STORES_BY_DEFAULT; " - "select history_source='agent_server' or 'service', or use store=true." - ) - if not stores_by_default and agent.default_options.get("store") is True: - raise RuntimeError( - "store=false with history_source='agent' cannot safely override a storing " - "default on a client that does not declare storage support." - ) - - request_context = get_request_context() - approval_storage = ( - self._function_approval_storage_provider.get_store(config=self.config, platform_context=request_context) - if stored - else None - ) - previous_response_id = request.get("previous_response_id") - session_load_id = ( - context.response_id - if context.is_recovery and provider_background - else context.conversation_id or previous_response_id - ) - if not stored and session_load_id is not None and self._history_source == "service": - raise ValueError( - "store=false cannot resume service-managed history; start a new one-shot request " - "or use history_source='agent_server' or 'agent'." - ) - session_storage = ( - self._session_storage_provider.get_store(config=self.config, platform_context=request_context) - if stored or (session_load_id is not None and self._history_source == "agent") - else None - ) - - # Load the caller's input items and prior conversation history concurrently with the - # session load below. These are independent storage round-trips with no data dependency - # between them, so overlapping them removes serial latency from the request critical path. - request_messages_task = asyncio.ensure_future( - self._load_request_messages( - context, - approval_storage=approval_storage, - configuration=configuration, - ) - ) - - session = ( - await session_storage.get(session_load_id) - if session_storage is not None and session_load_id is not None - else None - ) - if session is None: - if context.is_recovery and provider_background: - raise _PublicResponseFailure( - RuntimeError("Provider background was interrupted before its continuation token was stored.") - ) - if stored and previous_response_id is not None and context.conversation_id is None: - raise _PublicResponseFailure( - RuntimeError( - f"Cannot find an existing agent session for previous_response_id={previous_response_id}." - ) - ) - session = agent.create_session() - provider_state = session.state.get(_HOSTED_PROVIDER_STATE_KEY) - if context.conversation_id is not None: - if context.is_recovery and provider_background: - if session_storage is None: - raise RuntimeError("Provider background recovery requires agent session storage.") - head = await session_storage.get(context.conversation_id) - conversation_already_committed = ( - head is not None - and head.state.get(_HOSTED_CONVERSATION_CLAIM_KEY) is None - and head.state.get(_HOSTED_CONVERSATION_COMMITTED_KEY) == context.response_id - and isinstance(provider_state, Mapping) - and cast(Mapping[str, Any], provider_state).get("completed") is True - ) - if not conversation_already_committed and ( - head is None or head.state.get(_HOSTED_CONVERSATION_CLAIM_KEY) != context.response_id - ): - raise _PublicResponseFailure( - RuntimeError( - "Cannot recover provider background: the service-backed conversation claim " - "is no longer held by this response." - ) - ) - else: - _reject_busy_conversation(session) - if not stored and self._history_source == "agent" and session.service_session_id is not None: - raise _PublicResponseFailure( - ValueError( - "store=false cannot continue agent-managed downstream service history; " - "start a new one-shot request or use store=true." - ) - ) - if ( - not context.is_recovery - and provider_state is not None - and ( - not isinstance(provider_state, Mapping) - or cast(Mapping[str, Any], provider_state).get("completed") is not True - ) - ): - raise _PublicResponseFailure( - ValueError("A provider background response must complete before the next turn.") - ) - if previous_response_id is not None and context.conversation_id is None and not context.is_recovery: - if session.service_session_id is not None and session.state.get(_HOSTED_SOURCE_CONVERSATION_KEY): - raise _PublicResponseFailure( - ValueError("A service-managed downstream conversation cannot be forked.") - ) - if ( - stored - and self._history_source in ("service", "agent") - and session.service_session_id is not None - and session.state.get(_HOSTED_SERVICE_CHILD_KEY) - ): - raise _PublicResponseFailure(ValueError("A service-managed downstream response cannot be forked.")) - except BaseException as ex: - # Session preparation failed (or the request was cancelled / the stream closed — - # neither of which is an Exception). Cancel and drain the in-flight message-loading - # task so it is not orphaned, and log only ordinary failures. - if request_messages_task is not None: - request_messages_task.cancel() - with suppress(BaseException): - await request_messages_task - if isinstance(ex, Exception): - _log_response_failure("state storage preparation", context.response_id, ex) - raise - - request_failure: Exception | None = None - save_failure: Exception | None = None - request_interrupted = False - - try: - if configuration.agent_server_history: - session.state.pop(_HOSTED_RESPONSES_HISTORY_SOURCE_ID, None) - # A restored service ID belongs to the downstream model service. Replaying the - # AgentServer transcript while resuming that service history would duplicate every - # prior turn, so AgentServer-history mode always starts the model call statelessly. - session.service_session_id = None - - messages = await request_messages_task - run_kwargs: dict[str, Any] = { - "messages": messages, - "session": session, - } - chat_options = cast(ChatOptions[Any], dict(hosted_request.options)) - are_options_set = bool(chat_options) - if configuration.agent_server_history: - if configuration.client_stores_by_default: - # The response provider already owns the transcript used for this run. Keep a - # storing downstream service stateless so it cannot become a second history source. - chat_options["store"] = False - else: - # Do not pass a storage option to clients that do not advertise support for it. - chat_options.pop("store", None) - elif self._history_source == "service": - chat_options["store"] = stored - if not stored: - session.service_session_id = None - elif ( - not stored - and isinstance(agent, RawAgent) - and ( - getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", False) - or agent.default_options.get("store") is not None - ) - ): - chat_options["store"] = False - - if isinstance(agent, RawAgent): - run_kwargs["options"] = chat_options - elif are_options_set: - if self._unsupported_options == "error": - raise _PublicResponseFailure(TypeError("The hosted agent does not accept caller runtime options.")) - if self._unsupported_options == "warn": - logger.warning("Agent doesn't support runtime options. They will be ignored.") - - if previous_response_id is not None and context.conversation_id is None and not context.is_recovery: - if stored and self._history_source in ("service", "agent") and session.service_session_id is not None: - if session_storage is None: - raise RuntimeError("Service history requires agent session storage.") - session.state[_HOSTED_SERVICE_CHILD_KEY] = context.response_id - try: - await _set_agent_session(session_storage, previous_response_id, session) - finally: - session.state.pop(_HOSTED_SERVICE_CHILD_KEY, None) - session.state.pop(_HOSTED_SOURCE_CONVERSATION_KEY, None) - - if not context.is_recovery: - session.state.pop(_HOSTED_PROVIDER_STATE_KEY, None) - session.state.pop(_HOSTED_CONVERSATION_COMMITTED_KEY, None) - - if ( - stored - and context.conversation_id is not None - and not configuration.agent_server_history - and not conversation_already_committed - ): - if session_storage is None: - raise RuntimeError("Service history requires agent session storage.") - # The scoped store's conditional write must succeed before the provider can - # mutate its linear thread; a losing turn never reaches agent.run(). - session.state[_HOSTED_CONVERSATION_CLAIM_KEY] = context.response_id - try: - await _set_agent_session(session_storage, context.conversation_id, session) - finally: - session.state.pop(_HOSTED_CONVERSATION_CLAIM_KEY, None) - - inner_stream: ResponseStream[AgentResponseUpdate, AgentResponse[Any]] | None = None - updates: _SignalledIterator[AgentResponseUpdate] | None = None - if provider_background: - if session_storage is None or not isinstance(agent, RawAgent): - raise RuntimeError("Provider background requires a stored MAF agent session.") - output_count = _hosted_internal_metadata(response_event_stream).get( - _HOSTED_PROVIDER_OUTPUT_COUNT_KEY, 0 - ) - if type(output_count) is not int or output_count < 0: - raise RuntimeError("The persisted provider output cursor is invalid.") - responses = self._provider_background_responses( - agent=cast(RawAgent[ChatOptions[Any]], agent), - messages=messages, - session=session, - session_storage=session_storage, - options=chat_options, - context=context, - cancellation_signal=cancellation_signal, - emitted_output_count=output_count, - ) - async with aclosing(responses): - async for response, output_count in responses: - for update in _agent_response_updates(response, context.response_id): - if context.shutdown.is_set() or cancellation_signal.is_set(): - break - async for event in tracker.handle_update( - update, approval_storage=approval_storage, complete_message=True - ): - yield event - if context.shutdown.is_set() or cancellation_signal.is_set(): - continue - for event in tracker.close(): - yield event - private_metadata = _hosted_internal_metadata(response_event_stream) - private_metadata[_HOSTED_PROVIDER_OUTPUT_COUNT_KEY] = output_count - private_metadata[_HOSTED_PROVIDER_USAGE_KEY] = tracker.usage_details - if self._resilient_background: - yield response_event_stream.checkpoint() - else: - inner_stream = agent.run(stream=True, **run_kwargs) # type: ignore[reportUnknownMemberType] - updates = _SignalledIterator(inner_stream, context.shutdown, cancellation_signal) - async with aclosing(updates): - async for update in updates: - if not stored and any( - content.type in ("function_approval_request", "oauth_consent_request") - or content.user_input_request - for content in update.contents - ): - raise ValueError("Approval and user-input continuation requires store=true.") - async for event in tracker.handle_update(update, approval_storage=approval_storage): - yield event - if inner_stream is not None and isinstance(updates, _SignalledIterator) and not updates.signalled: - final = await inner_stream.get_final_response() - if final.continuation_token is not None and final.finish_reason is None: - raise RuntimeError( - "The inner agent returned an unfinished provider response; " - "configure background_source='provider' with history_source='service' to resume it." - ) - except (asyncio.CancelledError, GeneratorExit): - request_interrupted = True - raise - except Exception as ex: - request_failure = ex - _log_response_failure("agent execution", context.response_id, ex) - finally: - if configuration.hosted_history: - session.state.pop(_HOSTED_RESPONSES_HISTORY_SOURCE_ID, None) - if not provider_background and not context.is_recovery: - session.state.pop(_HOSTED_PROVIDER_STATE_KEY, None) - - # Never persist a session that could resume an inner response contrary to the - # history mode or the caller's explicit storage decision. - stored_output_violation = ( - self._history_source == "agent_server" or not stored - ) and session.service_session_id is not None - if stored_output_violation: - misconfigured = RuntimeError( - "The agent's chat client stored this turn server-side despite store=False for the inner model. " - "Configure the client to honor store=False, or use history_source='service' with store=true." - ) - _log_response_failure("stored output validation", context.response_id, misconfigured) - if request_failure is None and not request_interrupted: - request_failure = _PublicResponseFailure(misconfigured) - if ( - self._history_source == "service" - and stored - and _HOSTED_PROVIDER_STATE_KEY not in session.state - and session.service_session_id is None - and request_failure is None - and not request_interrupted - and not (cancellation_signal.is_set() and context.client_cancelled) - ): - request_failure = RuntimeError( - "history_source='service' requires the chat client to return a service continuation ID." - ) - try: - provider_state = session.state.get(_HOSTED_PROVIDER_STATE_KEY) - final_provider_state = not provider_background or ( - request_failure is None - and not request_interrupted - and not (cancellation_signal.is_set() and context.client_cancelled) - and ( - provider_state is None - or ( - isinstance(provider_state, Mapping) - and cast(Mapping[str, Any], provider_state).get("completed") is True - ) - ) - ) - superseded_by_steering = bool(self._host_options and self._host_options.steerable_conversations) and ( - cancellation_signal.is_set() and not context.client_cancelled and not context.shutdown.is_set() - ) - if stored and session_storage is not None and not stored_output_violation and final_provider_state: - if context.conversation_id is not None: - session.state[_HOSTED_SOURCE_CONVERSATION_KEY] = context.conversation_id - await _set_agent_session(session_storage, context.response_id, session) - if ( - context.conversation_id is not None - and not conversation_already_committed - and not superseded_by_steering - and not request_interrupted - and request_failure is None - and ( - configuration.agent_server_history - or (not cancellation_signal.is_set() and not context.shutdown.is_set()) - ) - ): - if provider_background: - session.state.pop(_HOSTED_PROVIDER_STATE_KEY, None) - session.state[_HOSTED_CONVERSATION_COMMITTED_KEY] = context.response_id - await _set_agent_session(session_storage, context.conversation_id, session) - except Exception as save_error: - save_failure = save_error - if request_interrupted: - message = "Failed to persist the Agent Framework session while unwinding an interrupted request" - elif request_failure is not None: - message = "Failed to persist the Agent Framework session after an agent failure" - else: - message = "Failed to persist the Agent Framework session after a successful request" - _log_response_failure(message, context.response_id, save_error) - - if request_failure is not None and save_failure is not None: - raise RuntimeError( - f"Agent request failed: {str(request_failure) or type(request_failure).__name__}; " - f"session persistence also failed: {str(save_failure) or type(save_failure).__name__}" - ) - elif request_failure is not None: - raise request_failure - elif save_failure is not None: - raise save_failure - - async def _provider_background_responses( - self, - *, - agent: RawAgent[ChatOptions[Any]], - messages: list[Message], - session: AgentSession, - session_storage: SessionStore, - options: ChatOptions[Any], - context: ResponseContext, - cancellation_signal: asyncio.Event, - emitted_output_count: int, - ) -> AsyncGenerator[tuple[AgentResponse[Any], int]]: - """Persist polling output with its private token and replay uncheckpointed output on recovery.""" - outputs: list[dict[str, Any]] = [] - - async def proceed(*, has_token: bool) -> bool: - if cancellation_signal.is_set() and context.client_cancelled: - if not has_token: - logger.warning( - "Provider background submission was cancelled before a token was saved; " - "remote work may continue." - ) - return False - if context.shutdown.is_set(): - if has_token and self._resilient_background: - await context.exit_for_recovery() - if has_token: - raise RuntimeError("Provider background recovery requires resilient_background=True.") - raise _PublicResponseFailure( - RuntimeError( - "Provider background submission stopped before its token was saved; cannot safely retry it." - ) - ) - if cancellation_signal.is_set(): - raise RuntimeError("Provider background was interrupted without a client cancellation.") - return True - - async def run_provider( - options: ChatOptions[Any], *, input_messages: list[Message] | None, phase: Literal["submit", "poll"] - ) -> AgentResponse[Any]: - try: - return await agent.run(input_messages, session=session, options=options) - except Exception as exc: - logger.warning("Inner provider background %s failed (%s).", phase, type(exc).__name__) - raise RuntimeError(f"Inner provider background {phase} failed; inspect the host logs.") - - async def save_private_state() -> None: - try: - await session_storage.set(context.response_id, session) - except Exception as exc: - logger.warning("Private provider background state save failed (%s).", type(exc).__name__) - else: - return - raise RuntimeError("Could not save private provider background state; inspect the host logs.") - - def record_output(response: AgentResponse[Any]) -> None: - messages = response.messages - if response.continuation_token is not None: - # A tool loop prefixes completed calls/results to the unfinished model - # response. Retain that prefix, not partial output that polling will repeat. - for index in range(len(messages) - 1, -1, -1): - message = messages[index] - last_result = next( - ( - offset - for offset in range(len(message.contents) - 1, -1, -1) - if message.contents[offset].type == "function_result" - ), - None, - ) - if last_result is not None: - completed_message = copy(message) - completed_message.contents = message.contents[: last_result + 1] - messages = [*messages[:index], completed_message] - break - else: - return - outputs.append( - AgentResponse( - messages=messages, - usage_details=response.usage_details, - finish_reason=FinishReason(response.finish_reason) if response.finish_reason is not None else None, - ).to_dict() - ) - - async def pending_outputs() -> AsyncGenerator[tuple[AgentResponse[Any], int]]: - nonlocal emitted_output_count - while emitted_output_count < len(outputs): - if not await proceed(has_token=True): - return - yield AgentResponse.from_dict(outputs[emitted_output_count]), emitted_output_count + 1 - if not await proceed(has_token=True): - return - emitted_output_count += 1 - - if context.is_recovery: - saved = session.state.get(_HOSTED_PROVIDER_STATE_KEY) - if not isinstance(saved, Mapping): - raise _PublicResponseFailure( - RuntimeError("Cannot recover a provider background job without its stored continuation token.") - ) - saved_payload = cast(Mapping[str, Any], saved) - if saved_payload.get("outer_response_id") != context.response_id: - raise _PublicResponseFailure( - RuntimeError("Cannot recover a provider background job without its stored continuation token.") - ) - token = saved_payload.get("continuation_token") - if not isinstance(token, Mapping): - raise RuntimeError("The stored provider continuation token is invalid.") - continuation_token: Mapping[str, Any] = cast(Mapping[str, Any], token) - saved_outputs = saved_payload.get("outputs", []) - if not isinstance(saved_outputs, list) or any( - not isinstance(output, dict) for output in cast(list[object], saved_outputs) - ): - raise RuntimeError("The stored provider output is invalid.") - outputs = cast(list[dict[str, Any]], saved_outputs) - if emitted_output_count > len(outputs): - raise RuntimeError("The persisted provider output cursor exceeds its private snapshot.") - async for output in pending_outputs(): - yield output - if saved_payload.get("completed") is True and outputs: - await proceed(has_token=True) - return - else: - if not await proceed(has_token=False): - return - completed, first = await _await_before_signal( - lambda: run_provider( - cast(ChatOptions[Any], {**options, "background": True}), - input_messages=messages, - phase="submit", - ), - context.shutdown, - cancellation_signal, - ) - if not completed: - if not await proceed(has_token=False): - return - raise RuntimeError("Provider background submission was interrupted before its token was saved.") - if first is None: - raise RuntimeError("The provider did not return a background response.") - if first.continuation_token is None: - if not await proceed(has_token=False): - return - yield first, 1 - return - if not isinstance(first.continuation_token, Mapping): - raise RuntimeError("The provider returned a continuation token that cannot be persisted.") - continuation_token = cast(Mapping[str, Any], first.continuation_token) - if any(message.contents for message in first.messages) or first.usage_details: - record_output(first) - session.state[_HOSTED_PROVIDER_STATE_KEY] = { - "outer_response_id": context.response_id, - "continuation_token": dict(continuation_token), - "outputs": outputs, - } - await save_private_state() - async for output in pending_outputs(): - yield output - - while True: - if not await proceed(has_token=True): - return - slept, _ = await _await_before_signal(lambda: asyncio.sleep(2), context.shutdown, cancellation_signal) - if not slept: - if not await proceed(has_token=True): - return - raise RuntimeError("Provider background polling was interrupted.") - if not await proceed(has_token=True): - return - completed, current = await _await_before_signal( - lambda token=continuation_token: run_provider( - cast( - ChatOptions[Any], - {**options, "background": True, "store": True, "continuation_token": token}, - ), - input_messages=None, - phase="poll", - ), - context.shutdown, - cancellation_signal, - ) - if not completed: - if not await proceed(has_token=True): - return - raise RuntimeError("Provider background polling was interrupted.") - if current is None: - raise RuntimeError("The provider did not return a background response.") - if current.continuation_token is None: - # Save the completed output with its token before publishing it; recovery - # replays the uncheckpointed output without invoking the agent again. - record_output(current) - session.state[_HOSTED_PROVIDER_STATE_KEY] = { - "outer_response_id": context.response_id, - "continuation_token": dict(continuation_token), - "completed": True, - "outputs": outputs, - } - await save_private_state() - async for output in pending_outputs(): - yield output - return - if not isinstance(current.continuation_token, Mapping): - raise RuntimeError("The provider returned a continuation token that cannot be persisted.") - if current.continuation_token == continuation_token: - continue - if any(message.contents for message in current.messages) or current.usage_details: - record_output(current) - continuation_token = cast(Mapping[str, Any], current.continuation_token) - session.state[_HOSTED_PROVIDER_STATE_KEY] = { - "outer_response_id": context.response_id, - "continuation_token": dict(continuation_token), - "outputs": outputs, - } - await save_private_state() - async for output in pending_outputs(): - yield output - - async def _handle_inner_workflow( - self, - request: CreateResponse, - context: ResponseContext, - response_event_stream: ResponseEventStream, - tracker: _OutputItemTracker, - cancellation_signal: asyncio.Event, - agent: WorkflowAgent, - ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: - """Handle the creation of a response for a workflow agent.""" - try: - request_context = get_request_context() - approval_storage = self._function_approval_storage_provider.get_store( - config=self.config, platform_context=request_context - ) - input_items = await context.get_input_items() - input_messages = await _items_to_messages(input_items, approval_storage=approval_storage) - - _, are_options_set = _to_chat_options(request) - if are_options_set: - logger.warning("Workflow agent doesn't support runtime options. They will be ignored.") - - # Determine the checkpoint storage for this request. The checkpoint - # storage is keyed by the conversation ID (if present) or the response - # ID (if no conversation ID is present). On a subsequent turn, the same - # conversation ID or a `previous_response_id` can be used to resume the - # workflow from the last checkpoint. - checkpoint_save_id = context.conversation_id or context.response_id - _validate_checkpoint_context_id(checkpoint_save_id) - checkpoint_storage = self._checkpoint_storage_provider.get_store( - config=self.config, - context_id=checkpoint_save_id, - platform_context=request_context, - ) - - if context.is_recovery: - if not self._resilient_background: - raise RuntimeError("Recovery mode is only supported when resilient_background=True.") - # Resume from the workflow checkpoint durably paired with the last persisted response - # snapshot (recorded in that snapshot's own metadata) -- NOT simply the latest workflow - # checkpoint in storage, which may be ahead of what response.output actually reflects if - # the crash happened between two response-stream checkpoint() calls. - checkpoint_id = _hosted_internal_metadata(response_event_stream).get(_LATEST_CHECKPOINT_ID_KEY) - if checkpoint_id is not None: - logger.debug("Serving recovery request from workflow checkpoint %s", checkpoint_id) - run_stream = self._resume_workflow_from_checkpoint( - checkpoint_id, checkpoint_storage, context.response_id, agent - ) - else: - latest_checkpoint = await checkpoint_storage.get_latest(workflow_name=agent.workflow.name) - if latest_checkpoint is not None: - logger.debug( - "Found a workflow checkpoint %s but no prior response snapshot was durably persisted; " - "resuming from the latest checkpoint", - latest_checkpoint.checkpoint_id, - ) - run_stream = self._resume_workflow_from_checkpoint( - latest_checkpoint.checkpoint_id, - checkpoint_storage, - context.response_id, - agent, - ) - else: - # No checkpoint was ever paired with a persisted response snapshot (e.g. the crash - # happened before the very first response checkpoint() call); replay the original - # input as a fresh entry, per the recovered-input parity guarantee - # (context.get_input_items() is unchanged from fresh entry). - logger.debug( - "Serving recovery request with no prior workflow checkpoint; replaying original input" - ) - run_stream = agent.run( - input_messages, - stream=True, - checkpoint_storage=checkpoint_storage, - ) - else: - # Determine the latest checkpoint (if any) so we can resume the - # workflow's prior state for this turn. The directory is keyed by - # the conversation id or the previous response id. - previous_response_id = request.get("previous_response_id") - if previous_response_id is not None and context.conversation_id is not None: - raise RuntimeError("Previous response ID cannot be used in conjunction with conversation ID.") - checkpoint_load_id = context.conversation_id or previous_response_id - restore_checkpoint_storage = checkpoint_storage - if checkpoint_load_id is not None: - _validate_checkpoint_context_id(checkpoint_load_id) - if checkpoint_load_id != checkpoint_save_id: - restore_checkpoint_storage = self._checkpoint_storage_provider.get_store( - config=self.config, - context_id=checkpoint_load_id, - platform_context=request_context, - ) - latest_checkpoint = await restore_checkpoint_storage.get_latest(workflow_name=agent.workflow.name) - - if latest_checkpoint is None and previous_response_id is not None: - # A previous_response_id must have a prior workflow checkpoint to resume from - raise _PublicResponseFailure( - RuntimeError( - "Cannot find an existing workflow checkpoint for " - f"previous_response_id={previous_response_id}." - ) - ) - - if latest_checkpoint is not None: - # If we have a prior checkpoint, restore it first (drive the workflow - # back to idle with prior state intact), then make a separate call that - # delivers the new user input. The restore-only call may yield events - # from any pending in-flight work in the checkpoint; we consume those - # internally here so they don't surface to the response stream as duplicates. - # - # If the restored checkpoint had pending request_info events, the - # restore-only call replays them through - # ``WorkflowAgent._convert_workflow_event_to_agent_response_updates`` - # and populates ``agent.pending_requests``. That is the correct - # state: those requests are genuinely outstanding, and the next - # ``run(input_messages, ...)`` call may contain ``function_call_output`` - # items (carried as FunctionResult/FunctionApprovalResponse content) - # that fulfill them via :meth:`WorkflowAgent._process_pending_requests`. - restore_iter = _SignalledIterator( - agent.run( - stream=True, - checkpoint_id=latest_checkpoint.checkpoint_id, - checkpoint_storage=restore_checkpoint_storage, - ), - context.shutdown, - cancellation_signal, - ) - async with aclosing(restore_iter): - async for _ in restore_iter: - pass - if restore_iter.signalled: - if context.shutdown.is_set(): - await context.exit_for_recovery() - if cancellation_signal.is_set(): - return - - # A cancel signal that fired after the restore-only replay finished (or was never - # entered) must still preempt starting a brand new workflow run below. - if cancellation_signal.is_set(): - return - - run_stream = agent.run( - input_messages, - stream=True, - checkpoint_storage=checkpoint_storage, - ) - - workflow_name = agent.workflow.name - - async def latest_checkpoint_id() -> str | None: - latest = await checkpoint_storage.get_latest(workflow_name=workflow_name) - return latest.checkpoint_id if latest is not None else None - - def snapshot_response( - checkpoint_id: str | None, - ) -> Generator[ResponseStreamEvent | ResponseCheckpointEvent]: - # Pair the response output emitted so far with the workflow checkpoint it corresponds - # to, so recovery from that checkpoint replays exactly the updates that came after it. - if checkpoint_id is None or checkpoint_id == _hosted_internal_metadata(response_event_stream).get( - _LATEST_CHECKPOINT_ID_KEY - ): - return - yield from tracker.close() - _hosted_internal_metadata(response_event_stream)[_LATEST_CHECKPOINT_ID_KEY] = checkpoint_id - yield response_event_stream.checkpoint() - - main_iter = _SignalledIterator( - run_stream, - context.shutdown, - cancellation_signal, - # The runner creates a checkpoint at the end of each superstep, inside the generator - # that produces the updates (see RunnerImpl.run_until_convergence). Stamping each - # update with the latest checkpoint as it was produced tells which checkpoint the - # update follows; the driver runs one update ahead, so by the time an update is - # consumed the workflow may already have checkpointed past it. - stamp=latest_checkpoint_id if self._resilient_background else None, - ) - async with aclosing(main_iter): - async for update in main_iter: - if self._resilient_background: - # Every update before this one belongs to the stamped checkpoint (or an - # earlier one), so the output so far can be snapshotted against it. If the - # workflow crashes before any update is produced, no snapshot is taken and - # recovery still resumes from the latest workflow checkpoint. - for event in snapshot_response(main_iter.stamp): - yield event - - async for event in tracker.handle_update(update, approval_storage=approval_storage): - yield event - # Cancellation needs no extra action here (the loop above already stopped); shutdown - # does, but only if it's what actually stopped the loop, not a natural completion. - if main_iter.signalled and context.shutdown.is_set(): - await context.exit_for_recovery() - elif self._resilient_background and not main_iter.signalled: - # The workflow ran to completion: pair its final checkpoint with the full output, so - # recovery after this point does not replay the last superstep. - for event in snapshot_response(await latest_checkpoint_id()): - yield event - except Exception as exc: - _log_response_failure("workflow execution", context.response_id, exc) - raise - - async def _resume_workflow_from_checkpoint( - self, - checkpoint_id: str, - checkpoint_storage: CheckpointStorage, - response_id: str, - agent: WorkflowAgent, - ) -> AsyncGenerator[AgentResponseUpdate]: - """Resume a crashed background workflow run, forwarding every event it produces. - - ``WorkflowAgent.run(checkpoint_id=..., messages=None)`` treats a message-less resume as - "restore only": it drives the workflow with the checkpoint's own already-queued internal - messages, but silently discards every event produced while doing so, on the assumption - that the workflow merely settles back to idle awaiting the next turn's input. That - assumption doesn't hold for crash recovery: the countdown (and any other self-driving - workflow) genuinely continues -- and may run to completion -- from its own queued - messages, and that output must not be lost. Drive the underlying ``Workflow`` directly so - none of it is discarded, converting each event the same way ``WorkflowAgent.run`` does. - - TODO(@taochen): #7677 - """ - role_gate = _StreamingRoleGate() - async for event in agent.workflow.run( - stream=True, - checkpoint_id=checkpoint_id, - checkpoint_storage=checkpoint_storage, - ): - for update in agent._convert_workflow_event_to_agent_response_updates( # pyright: ignore[reportPrivateUsage] - response_id, event, role_gate - ): - yield update - - @staticmethod - def _emit_failure( - response_id: str, - phase: str, - response_event_stream: ResponseEventStream, - tracker: _OutputItemTracker | None, - ex: BaseException, - *, - public_message: str | None = None, - ) -> Generator[ResponseStreamEvent]: - """Log ``ex`` and yield a terminal ``response.failed`` event. - - Drains any in-progress streaming output item first so the resulting - SSE stream stays well-formed, then emits ``response.failed`` carrying a - stable public message unless the caller explicitly supplies a safe - validation message. Any error raised while draining the tracker is - logged and otherwise ignored so that the original failure remains the - terminal failure. - """ - _log_response_failure(phase, response_id, ex) - if tracker is not None: - try: - yield from tracker.close() - except Exception as close_error: - _log_response_failure("failed tracker cleanup", response_id, close_error) - tracker.discard_pending_container_file_citations() - if public_message is not None: - message = public_message - elif isinstance(ex, _PublicResponseFailure): - message = ex.public_message - else: - message = _PUBLIC_STREAM_FAILURE_MESSAGE - yield response_event_stream.emit_failed(message=message, usage=tracker.usage if tracker is not None else None) - - -# endregion ResponsesHostServer - -# region Active Builder State - - -class _OutputItemTracker: - """Converts a stream of agent ``Content`` into ``ResponseStreamEvent``s for one response. - - For content types that arrive as a series of deltas (text, reasoning, function calls, MCP - calls) it tracks the single currently-open output item builder, merging consecutive same-item - deltas and closing the builder (emitting its `*_done` events) as soon as a different item - starts. All other content types (function results, image generation, shell calls/results, - approval requests, etc.) are emitted in one shot, closing any still-open streaming item first. - """ - - def __init__( - self, - stream: ResponseEventStream, - allowed_oauth_consent_origins: frozenset[str] | None = None, - ) -> None: - self._stream = stream - self._allowed_oauth_consent_origins = allowed_oauth_consent_origins - self._usage_details: UsageDetails | None = None - persisted_usage = _hosted_internal_metadata(stream).get(_HOSTED_PROVIDER_USAGE_KEY) - if persisted_usage is not None: - if not isinstance(persisted_usage, Mapping) or any( - not isinstance(key, str) or (value is not None and type(value) is not int) - for key, value in cast(Mapping[object, object], persisted_usage).items() - ): - raise RuntimeError("The persisted provider usage is invalid.") - self._usage_details = cast(UsageDetails, dict(cast(Mapping[str, int | None], persisted_usage))) - self._active_type: str | None = None - self._active_id: str | None = None - # message_id of the update that opened the active text item, used to detect a new - # logical message (e.g. a fresh workflow yield_output call) even when the content - # type doesn't change, so it isn't silently merged into the still-open item. - self._active_message_id: str | None = None - # Accumulated delta text for the current active builder - self._accumulated: list[str] = [] - # Builder state — only one is active at a time - self._message_item: OutputItemMessageBuilder | None = None - self._text_content: TextContentBuilder | None = None - self._refusal_content: RefusalContentBuilder | None = None - self._reasoning_item: OutputItemBuilder | None = None - self._summary_part: ReasoningSummaryPartBuilder | None = None - self._reasoning_encrypted_content: str | None = None - self._fc_builder: OutputItemFunctionCallBuilder | None = None - self._mcp_builder: OutputItemMcpCallBuilder | None = None - self._outstanding_function_calls: dict[str, str | None] = {} - self._outstanding_computer_calls: set[str] = set() - self._oauth_consent_requests: set[tuple[str, str]] = set() - self._pending_container_file_citations: dict[str, _ContainerFileCitation] = {} - self._text_annotations: list[ResponsesAnnotation] = [] - self._message_text_annotations: dict[int, list[ResponsesAnnotation]] = {} - # Set when an agent update reports the model stopped early (content filter, token - # limit); the response then ends as ``incomplete`` instead of ``completed`` so callers - # can tell a cut-short turn from a successful one. Mirrored onto the response so it - # survives a resilient checkpoint/recovery cycle, which rebuilds this tracker from the - # persisted response. - self._incomplete_reason: ResponseIncompleteReason | None = None - persisted_reason = _read_persisted_incomplete_reason(stream) - if persisted_reason is not None: - with suppress(ValueError): - self._incomplete_reason = ResponseIncompleteReason(persisted_reason) - persisted_citations = _hosted_internal_metadata(stream).get(_PENDING_CONTAINER_FILE_CITATIONS_KEY) - if persisted_citations is not None: - if not isinstance(persisted_citations, Sequence) or isinstance( - persisted_citations, (str, bytes, bytearray) - ): - raise RuntimeError("The persisted container file citations are invalid.") - for raw_citation in cast(Sequence[Any], persisted_citations): - if not isinstance(raw_citation, Mapping): - raise RuntimeError("The persisted container file citations are invalid.") - citation = cast(Mapping[str, Any], raw_citation) - container_id = citation.get("container_id") - file_id = citation.get("file_id") - filename = citation.get("filename") - if ( - not isinstance(container_id, str) - or not container_id - or not isinstance(file_id, str) - or not file_id - or not isinstance(filename, str) - or not filename - ): - raise RuntimeError("The persisted container file citations are invalid.") - self._pending_container_file_citations[filename] = _ContainerFileCitation( - container_id=container_id, - file_id=file_id, - filename=filename, - ) - for item in stream.response.get("output", []): - if not isinstance(item, Mapping): - continue - persisted_item = cast(Mapping[str, Any], item) - if persisted_item.get("type") == "computer_call": - call_id = persisted_item.get("call_id") - if isinstance(call_id, str): - self._outstanding_computer_calls.add(call_id) - elif persisted_item.get("type") == "computer_call_output": - if isinstance(call_id := persisted_item.get("call_id"), str): - self._outstanding_computer_calls.discard(call_id) - if persisted_item.get("type") != "oauth_consent_request": - continue - consent_link = persisted_item.get("consent_link") - server_label = persisted_item.get("server_label") - if isinstance(consent_link, str) and isinstance(server_label, str): - self._oauth_consent_requests.add((consent_link, server_label)) - - @property - def usage_details(self) -> UsageDetails | None: - """Return usage retained with the provider output checkpoint.""" - return self._usage_details - - @property - def usage(self) -> ResponseUsage | None: - """Return accumulated usage in the Responses API schema.""" - if self._usage_details is None: - return None - - input_tokens = int(self._usage_details.get("input_token_count") or 0) - output_tokens = int(self._usage_details.get("output_token_count") or 0) - total_tokens = self._usage_details.get("total_token_count") - return ResponseUsage( - input_tokens=input_tokens, - input_tokens_details=ResponseUsageInputTokensDetails( - cached_tokens=int(self._usage_details.get("cache_read_input_token_count") or 0), - cache_write_tokens=int(self._usage_details.get("cache_creation_input_token_count") or 0), - ), - output_tokens=output_tokens, - output_tokens_details=ResponseUsageOutputTokensDetails( - reasoning_tokens=int(self._usage_details.get("reasoning_output_token_count") or 0) - ), - total_tokens=int(total_tokens) if total_tokens is not None else input_tokens + output_tokens, - ) - - @property - def oauth_consent_requested(self) -> bool: - """Return whether this response emitted an OAuth consent request.""" - return bool(self._oauth_consent_requests) - - @property - def incomplete_reason(self) -> ResponseIncompleteReason | None: - """Return why the turn was cut short, if any update reported a truncating finish reason.""" - return self._incomplete_reason - - def record_finish_reason(self, finish_reason: str | None) -> None: - """Note the finish reason of an agent update. - - Only finish reasons that mean the model stopped early are retained, mapped onto the - Responses ``incomplete_details.reason`` vocabulary. A content filter is kept in - preference to a token limit if both are seen during a multi-step turn, since it is the - more actionable signal for the caller. The reason is recorded even when the caller's - metadata already fills the public map, and that map is left intact. - """ - if finish_reason == "content_filter": - self._incomplete_reason = ResponseIncompleteReason.CONTENT_FILTER - elif finish_reason == "length" and self._incomplete_reason is None: - self._incomplete_reason = ResponseIncompleteReason.MAX_OUTPUT_TOKENS - else: - return - _hosted_internal_metadata(self._stream)[_INCOMPLETE_REASON_KEY] = self._incomplete_reason.value - - def _consume_text_update(self, text: str, merge_mode: Literal["append", "replace"] | None) -> str: - """Update the authoritative text buffer and return the wire delta still needed.""" - if merge_mode != "replace": - self._accumulated.append(text) - return text - - previous = "".join(self._accumulated) - if text.startswith(previous): - self._accumulated[:] = [text] - return text[len(previous) :] - if not previous: - self._accumulated[:] = [text] - return text - raise _PublicResponseFailure( - RuntimeError("Cannot project a divergent replacement text snapshot onto an append-only Responses stream.") - ) - - async def handle_update( - self, - update: AgentResponseUpdate, - *, - approval_storage: FunctionApprovalStore | None = None, - complete_message: bool = False, - ) -> AsyncGenerator[ResponseStreamEvent]: - """Process one agent update: note its finish reason, then handle each of its contents. - - This is the single entry point for both the plain-agent and the workflow loops, so the - finish reason cannot be forgotten on one of them. Set ``complete_message`` when the - update contains a completed message's independent content parts rather than streaming - deltas; text and refusal parts then retain their original boundaries and citation offsets. - """ - self.record_finish_reason(update.finish_reason) - for content in update.contents: - async for event in self.handle( - content, - message_id=update.message_id, - approval_storage=approval_storage, - start_new_content_part=complete_message, - ): - yield event - - async def handle( - self, - content: Content, - message_id: str | None = None, - *, - approval_storage: FunctionApprovalStore | None = None, - start_new_content_part: bool = False, - ) -> AsyncGenerator[ResponseStreamEvent]: - """Process a content item, yielding its events. - - Args: - content: The content item to process. - message_id: The ``message_id`` of the update ``content`` came from, if any. A - change in ``message_id`` across otherwise same-typed text content marks a new - logical message and forces the previous output item closed, rather than being - merged into it. - approval_storage: Used for content types that fall back to one-shot emission - (anything not recognized as a streaming delta type) to save/load approval requests. - start_new_content_part: Whether text and refusal content start a new independent - content part instead of continuing the current streaming occurrence. - """ - if _is_refusal_text_content(content) and content.text is not None: - for event in self._ensure_message_content("refusal", message_id, start_new_part=start_new_content_part): - yield event - self._active_message_id = message_id - delta = self._consume_text_update(content.text, content.merge_mode) - if delta and self._refusal_content is not None: - yield self._refusal_content.emit_delta(delta) - - elif content.type == "text" and content.text is not None: - for event in self._ensure_message_content("text", message_id, start_new_part=start_new_content_part): - yield event - self._active_message_id = message_id - delta = self._consume_text_update(content.text, content.merge_mode) - for annotation in _annotations_to_output_text(content.annotations): - if annotation not in self._text_annotations: - self._text_annotations.append(annotation) - if delta and self._text_content is not None: - yield self._text_content.emit_delta(delta) - - elif content.type == "text_reasoning": - if self._active_type != "text_reasoning" or (content.id is not None and content.id != self._active_id): - for event in self._close(): - yield event - for event in self._open_reasoning(content): - yield event - if encrypted_content := _reasoning_encrypted_content(content): - self._reasoning_encrypted_content = encrypted_content - if content.text: - delta = self._consume_text_update(content.text, content.merge_mode) - if delta and self._summary_part is not None: - yield self._summary_part.emit_text_delta(delta) - - elif content.type == "function_call" and content.call_id is not None: - # Declaration-only calls replay request metadata after the streamed call. Scope suppression to the - # outstanding occurrence because a call_id may be reused after its terminal result. - if ( - content.user_input_request - and content.arguments is None - and content.call_id in self._outstanding_function_calls - and self._outstanding_function_calls[content.call_id] == content.name - ): - return - if self._active_type != "function_call" or self._active_id != content.call_id: - for event in self._close(): - yield event - for event in self._open_function_call(content): - yield event - args_str = _json_safe_to_str(content.arguments) - delta = self._consume_text_update(args_str, content.merge_mode) - if delta and self._fc_builder is not None: - yield self._fc_builder.emit_arguments_delta(delta) - - elif content.type == "function_result": - for event in self._close(): - yield event - citations = _container_file_citations_from_function_result(content) - if citations: - for citation in citations: - self._pending_container_file_citations[citation.filename] = citation - self._persist_pending_container_file_citations() - async for event in self._stream.output_item_function_call_output( - content.call_id, # type: ignore[arg-type] - _json_safe_to_str(content.result), - ): - yield event - if content.call_id is not None: - self._outstanding_function_calls.pop(content.call_id, None) - - elif content.type == "mcp_server_tool_call" and content.tool_name: - key = content.call_id or f"{content.server_name or 'default'}::{content.tool_name}" - if self._active_type != "mcp_server_tool_call" or self._active_id != key: - for event in self._close(): - yield event - for event in self._open_mcp_call(content): - yield event - args_str = _json_safe_to_str(content.arguments) - delta = self._consume_text_update(args_str, content.merge_mode) - if delta and self._mcp_builder is not None: - yield self._mcp_builder.emit_arguments_delta(delta) - - elif ( - content.type == "mcp_server_tool_result" - and self._active_type == "mcp_server_tool_call" - and self._mcp_builder is not None - and content.call_id is not None - and content.call_id == self._mcp_builder.item_id - ): - accumulated = "".join(self._accumulated) - yield self._mcp_builder.emit_arguments_done(accumulated) - yield self._mcp_builder.emit_completed() - yield self._mcp_builder.emit_done(output=_stringify_mcp_output(content.output)) - self._mcp_builder = None - self._active_type = None - self._active_id = None - self._accumulated.clear() - return - - elif content.type == "image_generation_tool_result" and content.outputs is not None: - for event in self._close(): - yield event - result = _image_generation_result(content.outputs) - if result is None: - logger.warning("Image generation result has no base64 image; emitting it with an empty result.") - async for event in self._stream.output_item_image_gen_call(result or ""): - yield event - - elif content.type == "mcp_server_tool_call": - # Reached only when `content.tool_name` is falsy (the streaming branch above didn't match). - for event in self._close(): - yield event - mcp_call = self._stream.add_output_item_mcp_call( - server_label=content.server_name or "default", - name=content.tool_name or "", - item_id=content.call_id, - ) - yield mcp_call.emit_added() - async for event in mcp_call.arguments(_json_safe_to_str(content.arguments)): - yield event - yield mcp_call.emit_completed() - yield mcp_call.emit_done() - - elif content.type == "mcp_server_tool_result": - # Reached when there's no correlated in-progress mcp_server_tool_call to close against. - for event in self._close(): - yield event - output = _stringify_mcp_output(content.output) - async for event in self._stream.output_item_custom_tool_call_output(content.call_id or "", output): - yield event - - elif content.type == "shell_tool_call": - for event in self._close(): - yield event - action = FunctionShellAction( - commands=content.commands or [], - timeout_ms=content.timeout_ms, - max_output_length=content.max_output_length, - ) - async for event in self._stream.output_item_function_shell_call( - content.call_id or "", - action, - LocalEnvironmentResource(type="local"), - status=content.status or "completed", - ): - yield event - - elif content.type == "shell_tool_result": - for event in self._close(): - yield event - output_items: list[FunctionShellCallOutputContent] = [] - if content.outputs: - for out in content.outputs: - exit_code = getattr(out, "exit_code", None) - outcome = ( - FunctionShellCallOutputTimeoutOutcome(type="timeout") - if getattr(out, "timed_out", False) - else FunctionShellCallOutputExitOutcome( - type="exit", - exit_code=exit_code if exit_code is not None else 0, - ) - ) - output_items.append( - FunctionShellCallOutputContent( - stdout=getattr(out, "stdout", "") or "", - stderr=getattr(out, "stderr", "") or "", - outcome=outcome, - ) - ) - async for event in self._stream.output_item_function_shell_call_output( - content.call_id or "", - output_items, - status=content.status or "completed", - max_output_length=content.max_output_length, - ): - yield event - - elif content.type == "computer_tool_call": - if not content.id or not content.call_id or not content.actions: - raise ValueError("A computer call requires an item id, call_id, and actions.") - if content.call_id in self._outstanding_computer_calls: - return - for event in self._close(): - yield event - if IdGenerator.is_valid(content.id)[0]: - builder = self._stream.add_output_item(content.id) - else: - logger.warning("Remapping computer call item id %r to an AgentServer id.", content.id) - builder = self._stream.add_output_item_computer_call() - status = content.status if content.status is not None else "completed" - if status not in ("in_progress", "completed", "incomplete"): - raise ValueError(f"Unsupported computer call status: {status!r}") - item = OutputItemComputerToolCall( - type="computer_call", - id=builder.item_id, - call_id=content.call_id, - actions=cast(list[ComputerAction], content.actions), - pending_safety_checks=cast(list[ComputerCallSafetyCheckParam], content.pending_safety_checks or []), - status=status, - ) - if _openai_value(content, "computer_action_format", "computer_action_format") == "single": - if len(content.actions) != 1: - raise ValueError("A preview computer call requires exactly one action.") - item.pop("actions") - item["action"] = cast(ComputerAction, content.actions[0]) - yield builder.emit_added(item) - yield builder.emit_done(item) - self._outstanding_computer_calls.add(content.call_id) - - elif content.type == "computer_tool_result": - if not content.call_id or content.screenshot is None: - raise _PublicResponseFailure(ValueError("A computer result requires a call_id and screenshot.")) - for event in self._close(): - yield event - if content.id and IdGenerator.is_valid(content.id)[0]: - builder = self._stream.add_output_item(content.id) - else: - if content.id: - logger.warning("Remapping computer output item id %r to an AgentServer id.", content.id) - builder = self._stream.add_output_item_computer_call_output() - item = OutputItemComputerToolCallOutput( - type="computer_call_output", - id=builder.item_id, - call_id=content.call_id, - output=cast(ComputerScreenshotImage, _computer_screenshot_to_output(content.screenshot)), - ) - if content.status is not None: - if content.status not in ("in_progress", "completed", "incomplete"): - raise ValueError(f"Unsupported computer output status: {content.status!r}") - item["status"] = content.status - if content.acknowledged_safety_checks is not None: - item["acknowledged_safety_checks"] = cast( - list[ComputerCallSafetyCheckParam], content.acknowledged_safety_checks - ) - yield builder.emit_added(item) - yield builder.emit_done(item) - self._outstanding_computer_calls.discard(content.call_id) - - elif content.type == "function_approval_request": - for event in self._close(): - yield event - function_call: Content = content.function_call # type: ignore - server_label = function_call.server_name or "agent_framework" - request_saved = False - async for event in self._stream.output_item_mcp_approval_request( - server_label, - function_call.name, # type: ignore - _json_safe_to_str(function_call.arguments), - ): - if approval_storage is not None and not request_saved: - # Extract the approval request ID generated by the infrastructure when the - # approval request item is added to the stream, and save it to approval - # storage so it can be retrieved later for round trips. - item = event.get("item") if isinstance(event, Mapping) else getattr(event, "item", None) - approval_request_id = ( - cast(Mapping[str, Any], item).get("id") - if isinstance(item, Mapping) - else getattr(item, "id", None) - ) - if isinstance(approval_request_id, str): - await approval_storage.save_approval_request(approval_request_id, content) - request_saved = True - yield event - if approval_storage is not None and not request_saved: - logger.warning( - "Approval request was not saved to approval storage because the approval request ID " - "could not be extracted from the stream event." - ) - - elif content.type == "oauth_consent_request": - for event in self._close(): - yield event - - consent_link = content.consent_link - if not _is_allowed_oauth_consent_link(consent_link, self._allowed_oauth_consent_origins): - raise ValueError("OAuth consent request content must include an allowed safe HTTPS consent link.") - - server_label = get_namespaced_value( - content.additional_properties, - "openai", - "server_label", - content.additional_properties.get("server_label"), - ) - if not isinstance(server_label, str) or not server_label: - server_label = getattr(content.raw_representation, "server_label", None) - if not isinstance(server_label, str) or not server_label: - server_label = "agent_framework" - - consent_key = (consent_link, server_label) - if consent_key in self._oauth_consent_requests: - return - self._oauth_consent_requests.add(consent_key) - - oauth_item = OAuthConsentRequestOutputItem( - id=IdGenerator.new_id("oacr"), - response_id=str(self._stream.response["id"]), - type="oauth_consent_request", - consent_link=consent_link, - server_label=server_label, - ) - builder = self._stream.add_output_item(oauth_item["id"]) - yield builder.emit_added(oauth_item) - yield builder.emit_done(oauth_item) - - elif content.type == "usage": - self._usage_details = add_usage_details(self._usage_details, content.usage_details) - - else: - for event in self._close(): - yield event - # Defensive: covers content types not recognized above (e.g. "text"/"text_reasoning"/ - # "function_call" with missing required fields), logged instead of raised so the - # response stream isn't broken by one unsupported content item. - logger.warning(f"Content type '{content.type}' is not supported yet. This is usually safe to ignore.") - - def close(self) -> Generator[ResponseStreamEvent]: - """Flush any remaining active builder without discarding recoverable state.""" - yield from self._close() - - def discard_pending_container_file_citations(self) -> None: - """Discard unmatched citations immediately before a terminal response event.""" - self._pending_container_file_citations.clear() - self._persist_pending_container_file_citations() - - # -- Private open/close helpers -- - - def _ensure_message_content( - self, - content_type: Literal["text", "refusal"], - message_id: str | None, - *, - start_new_part: bool = False, - ) -> Generator[ResponseStreamEvent]: - message_changed = ( - message_id is not None and self._active_message_id is not None and message_id != self._active_message_id - ) - if self._active_type == content_type and not message_changed and not start_new_part: - return - if self._message_item is not None and self._active_type in {"text", "refusal"} and not message_changed: - yield from self._close_message_content() - yield from self._open_message_content(content_type) - return - yield from self._close() - yield from self._open_message(content_type) - - def _open_message(self, content_type: Literal["text", "refusal"]) -> Generator[ResponseStreamEvent]: - self._message_text_annotations.clear() - self._message_item = self._stream.add_output_item_message() - yield self._message_item.emit_added() - yield from self._open_message_content(content_type) - - def _open_message_content( - self, - content_type: Literal["text", "refusal"], - ) -> Generator[ResponseStreamEvent]: - if self._message_item is None: - raise RuntimeError("Cannot open message content without an active message") - self._active_type = content_type - self._active_id = None - if content_type == "refusal": - self._refusal_content = self._message_item.add_refusal_content() - yield self._refusal_content.emit_added() - else: - self._text_content = self._message_item.add_text_content() - self._text_annotations.clear() - yield self._text_content.emit_added() - - def _open_reasoning(self, content: Content) -> Generator[ResponseStreamEvent]: - item_id = content.id - if not item_id or not IdGenerator.is_valid(item_id)[0]: - item_id = IdGenerator.new_id("rs") - self._reasoning_item = self._stream.add_output_item(item_id) - self._summary_part = ReasoningSummaryPartBuilder( - self._stream, - self._reasoning_item.output_index, - 0, - item_id, - ) - self._reasoning_encrypted_content = _reasoning_encrypted_content(content) - self._active_type = "text_reasoning" - self._active_id = item_id - yield self._reasoning_item.emit_added( - _reasoning_output_item( - item_id=item_id, - summary_texts=[], - encrypted_content=None, - status="in_progress", - ) - ) - yield self._summary_part.emit_added() - - def _open_function_call(self, content: Content) -> Generator[ResponseStreamEvent]: - self._fc_builder = self._stream.add_output_item_function_call( - name=content.name or "", - call_id=content.call_id or "", - ) - self._active_type = "function_call" - self._active_id = content.call_id - self._outstanding_function_calls[content.call_id or ""] = content.name - yield self._fc_builder.emit_added() - - def _open_mcp_call(self, content: Content) -> Generator[ResponseStreamEvent]: - self._mcp_builder = self._stream.add_output_item_mcp_call( - server_label=content.server_name or "default", - name=content.tool_name or "", - item_id=content.call_id, - ) - self._active_type = "mcp_server_tool_call" - self._active_id = content.call_id or f"{content.server_name or 'default'}::{content.tool_name}" - yield self._mcp_builder.emit_added() - - def _close(self) -> Generator[ResponseStreamEvent]: - if self._active_type in {"text", "refusal"}: - yield from self._close_message_content() - if self._message_item is not None: - message_done = self._message_item.emit_done() - if self._message_text_annotations: - message_done_dict = cast(dict[str, Any], message_done) - item = cast(dict[str, Any], message_done_dict["item"]) - content_parts = cast(list[dict[str, Any]], item["content"]) - for content_index, annotations in self._message_text_annotations.items(): - if content_index >= len(content_parts): - raise RuntimeError("Container file citation content index is out of range.") - content_parts[content_index]["annotations"] = annotations - - response_output = self._stream.response.get("output") - output_index = self._message_item.output_index - if not isinstance(response_output, list): - raise RuntimeError("Container file citation output index is out of range.") - response_output_items = cast(list[Any], response_output) - if output_index >= len(response_output_items): - raise RuntimeError("Container file citation output index is out of range.") - response_output_items[output_index] = item - yield message_done - self._message_item = None - self._message_text_annotations.clear() - - elif self._active_type == "text_reasoning" and self._summary_part and self._reasoning_item: - accumulated = "".join(self._accumulated) - yield self._summary_part.emit_text_done(accumulated) - yield self._summary_part.emit_done() - yield self._reasoning_item.emit_done( - _reasoning_output_item( - item_id=self._reasoning_item.item_id, - summary_texts=[accumulated], - encrypted_content=self._reasoning_encrypted_content, - status="completed", - ) - ) - self._summary_part = None - self._reasoning_item = None - self._reasoning_encrypted_content = None - - elif self._active_type == "function_call" and self._fc_builder: - accumulated = "".join(self._accumulated) - yield self._fc_builder.emit_arguments_done(accumulated) - yield self._fc_builder.emit_done() - self._fc_builder = None - - elif self._active_type == "mcp_server_tool_call" and self._mcp_builder: - accumulated = "".join(self._accumulated) - yield self._mcp_builder.emit_arguments_done(accumulated) - yield self._mcp_builder.emit_completed() - yield self._mcp_builder.emit_done() - self._mcp_builder = None - - self._active_type = None - self._active_id = None - self._active_message_id = None - self._accumulated.clear() - - def _close_message_content(self) -> Generator[ResponseStreamEvent]: - accumulated = "".join(self._accumulated) - if self._active_type == "text" and self._text_content is not None: - annotations = list(self._text_annotations) - for annotation in self._container_file_annotations_for_text(accumulated): - if annotation not in annotations: - annotations.append(annotation) - yield self._text_content.emit_text_done(accumulated) - for annotation in annotations: - yield self._text_content.emit_annotation_added(annotation) - content_done = self._text_content.emit_done() - if annotations: - content_done_dict = cast(dict[str, Any], content_done) - part = cast(dict[str, Any], content_done_dict["part"]) - part["annotations"] = annotations - self._message_text_annotations[self._text_content.content_index] = annotations - yield content_done - self._text_content = None - self._text_annotations.clear() - elif self._active_type == "refusal" and self._refusal_content is not None: - yield self._refusal_content.emit_refusal_done(accumulated) - yield self._refusal_content.emit_done() - self._refusal_content = None - self._active_type = None - self._active_id = None - self._accumulated.clear() - - def _container_file_annotations_for_text(self, text: str) -> list[ContainerFileCitationBody]: - candidates: list[tuple[int, int, str, _ContainerFileCitation]] = [] - for filename, citation in self._pending_container_file_citations.items(): - search_index = 0 - while True: - start_index = text.find(filename, search_index) - if start_index < 0: - break - end_index = start_index + len(filename) - if _has_container_filename_boundaries(text, start_index, end_index): - candidates.append((start_index, end_index, filename, citation)) - search_index = start_index + 1 - - candidates.sort(key=lambda match: (-(match[1] - match[0]), match[0], match[2])) - selected: list[tuple[int, int, str, _ContainerFileCitation]] = [] - selected_filenames: set[str] = set() - for candidate in candidates: - start_index, end_index, filename, _ = candidate - if filename in selected_filenames: - continue - if any( - start_index < selected_end and selected_start < end_index - for selected_start, selected_end, _, _ in selected - ): - continue - selected.append(candidate) - selected_filenames.add(filename) - - annotations: list[ContainerFileCitationBody] = [] - for start_index, end_index, filename, citation in sorted(selected, key=lambda match: match[0]): - annotations.append( - ContainerFileCitationBody( - type="container_file_citation", - container_id=citation.container_id, - file_id=citation.file_id, - filename=citation.filename, - start_index=start_index, - end_index=end_index, - ) - ) - del self._pending_container_file_citations[filename] - if annotations: - self._persist_pending_container_file_citations() - return annotations - - def _persist_pending_container_file_citations(self) -> None: - if not self._pending_container_file_citations: - _hosted_internal_metadata(self._stream).pop(_PENDING_CONTAINER_FILE_CITATIONS_KEY, None) - return - _hosted_internal_metadata(self._stream)[_PENDING_CONTAINER_FILE_CITATIONS_KEY] = [ - { - "container_id": citation.container_id, - "file_id": citation.file_id, - "filename": citation.filename, - } - for citation in self._pending_container_file_citations.values() - ] - - -# endregion - - -# region Option Conversion - - -def _to_chat_options(request: CreateResponse) -> tuple[ChatOptions, bool]: - """Converts a CreateResponse request to ChatOptions. - - Args: - request (CreateResponse): The request to convert. - - Returns: - ChatOptions: The converted ChatOptions. - bool: Whether any options were set. - - """ - chat_options = ChatOptions() - are_options_set = False - - if (temperature := request.get("temperature")) is not None: - chat_options["temperature"] = temperature - are_options_set = True - if (top_p := request.get("top_p")) is not None: - chat_options["top_p"] = top_p - are_options_set = True - if (max_output_tokens := request.get("max_output_tokens")) is not None: - chat_options["max_tokens"] = max_output_tokens - are_options_set = True - if (parallel_tool_calls := request.get("parallel_tool_calls")) is not None: - chat_options["allow_multiple_tool_calls"] = parallel_tool_calls - are_options_set = True - - return chat_options, are_options_set - - -# endregion - - -# region Input Message Conversion - - -async def _items_to_messages( - input_items: Sequence[Item], *, approval_storage: FunctionApprovalStore | None = None -) -> list[Message]: - """Converts a sequence of input items to a list of Messages, one per item. - - Args: - input_items: The input items to convert. - approval_storage: An optional ApprovalStorage instance used to look up - approval requests when converting MCP approval response items. - - Returns: - A list of Messages, one per supported input item. - """ - messages: list[Message] = [] - for item in input_items: - messages.append(await _item_to_message(item, approval_storage=approval_storage)) - return messages - - -def _reasoning_item_to_contents(reasoning: ItemReasoningItem | OutputItemReasoningItem) -> list[Content]: - """Convert a hosted reasoning item without losing its stateless replay metadata.""" - encrypted_content = reasoning.get("encrypted_content") - if summary_parts := reasoning.get("summary"): - return [ - Content.from_text_reasoning( - id=reasoning["id"], - text=summary["text"], - protected_data=encrypted_content if index == 0 else None, - ) - for index, summary in enumerate(summary_parts) - ] - return [Content.from_text_reasoning(id=reasoning["id"], protected_data=encrypted_content)] - - -def _computer_safety_checks(checks: Sequence[Mapping[str, Any]] | None) -> list[ComputerSafetyCheck] | None: - if checks is None: - return None - parsed: list[ComputerSafetyCheck] = [] - for check in checks: - check_id = check.get("id") - if not isinstance(check_id, str) or not check_id: - raise ValueError("Computer safety checks require an id.") - entry = ComputerSafetyCheck(id=check_id) - for key in ("code", "message"): - value = check.get(key) - if value is not None: - if not isinstance(value, str): - raise ValueError(f"Computer safety check {key} must be a string.") - entry[key] = value - parsed.append(entry) - return parsed - - -def _computer_screenshot_from_output(output: Mapping[str, Any]) -> Content: - if output.get("type") != "computer_screenshot": - raise ValueError("Computer call output must contain a computer screenshot.") - image_url = output.get("image_url") - file_id = output.get("file_id") - openai_properties: dict[str, Any] = {} - legacy_properties: dict[str, Any] = {} - if detail := output.get("detail"): - openai_properties["image_detail"] = detail - legacy_properties["detail"] = detail - additional_properties = { - "openai": openai_properties, - **legacy_properties, - } - if isinstance(image_url, str) and image_url: - return Content.from_uri( - image_url, - file_id=file_id if isinstance(file_id, str) else None, - additional_properties={ - **additional_properties, - **({"file_id": file_id} if isinstance(file_id, str) else {}), - }, - ) - if isinstance(file_id, str) and file_id: - return Content.from_hosted_file(file_id, additional_properties=additional_properties) - raise ValueError("Computer screenshot is missing its image URL or file ID.") - - -def _computer_screenshot_to_output(screenshot: Content) -> dict[str, Any]: - output: dict[str, Any] = {"type": "computer_screenshot"} - if screenshot.type in ("data", "uri") and screenshot.uri: - output["image_url"] = screenshot.uri - if file_id := screenshot.file_id or screenshot.additional_properties.get("file_id"): - output["file_id"] = file_id - elif screenshot.type == "hosted_file" and screenshot.file_id: - output["file_id"] = screenshot.file_id - else: - raise ValueError("A computer screenshot requires image data, a URI, or a hosted file.") - if detail := _openai_value(screenshot, "image_detail", "detail"): - output["detail"] = detail - return output - - -def _shell_command_output_to_content(output: Mapping[str, Any]) -> Content: - outcome = output["outcome"] - outcome_type = outcome.get("type") - return Content.from_shell_command_output( - stdout=output.get("stdout") or "", - stderr=output.get("stderr") or "", - exit_code=outcome.get("exit_code") if outcome_type == "exit" else None, - timed_out=True if outcome_type == "timeout" else False if outcome_type == "exit" else None, - ) - - -async def _item_to_message( - item: Item, - *, - approval_storage: FunctionApprovalStore | None = None, - _item_type_name: Literal["Item", "OutputItem"] = "Item", -) -> Message: - """Converts an Item to a Message. - - Args: - item: The Item to convert. - approval_storage: An optional ApprovalStorage instance used to look up - approval requests when converting MCP approval response items. - _item_type_name: The item type name to include in unsupported-type errors. - - Returns: - The converted Message. - - Raises: - ValueError: If the Item type is not supported. - """ - if item["type"] == "message": - if isinstance(item["content"], str): - return Message(role=item["role"], contents=[Content.from_text(item["content"])]) - return Message(role=item["role"], contents=[_convert_message_content(part) for part in item["content"]]) - - if item["type"] == "output_message": - return Message(role=item["role"], contents=[_convert_output_message_content(part) for part in item["content"]]) - - if item["type"] == "function_call": - item_data = cast("Mapping[str, Any]", item) - item_id = item_data.get("id") - status = item_data.get("status") - additional_properties: dict[str, Any] = {} - if isinstance(item_id, str): - additional_properties.update({"openai": {"item_id": item_id}, "fc_id": item_id}) - if status is not None: - additional_properties["status"] = status - return Message( - role="assistant", - contents=[ - Content.from_function_call( - item["call_id"], - item["name"], - arguments=item["arguments"], - status=status if isinstance(status, str) else None, - additional_properties=additional_properties, - ) - ], - ) - - if item["type"] == "function_call_output": - call_id = item.get("call_id") - if call_id is None: - raise ValueError("Function call output item is missing a call_id.") - item_id = item.get("id") - result_additional_properties = ( - { - "openai": {"item_id": item_id}, - "item_id": item_id, - } - if isinstance(item_id, str) and item_id - else None - ) - return Message( - role="tool", - contents=[ - Content.from_function_result( - call_id, - result=_json_safe_to_str(item["output"]), - status=item.get("status"), - additional_properties=result_additional_properties, - ) - ], - ) - - if item["type"] == "reasoning": - return Message(role="assistant", contents=_reasoning_item_to_contents(item)) - - if item["type"] == "mcp_call": - contents = [ - Content.from_mcp_server_tool_call( - item["id"], - item["name"], - server_name=item["server_label"], - arguments=item["arguments"], - ) - ] - if (output := item.get("output")) is not None: - contents.append(Content.from_mcp_server_tool_result(call_id=item["id"], output=output)) - return Message( - role="assistant", - contents=contents, - ) - - if item["type"] == "mcp_approval_request": - if approval_storage is not None: - try: - function_approval_request_content = await approval_storage.load_approval_request(item["id"]) - except (KeyError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - else: - raise ValueError("ApprovalStorage is required to load approval request.") - return Message( - role="assistant", - contents=[function_approval_request_content], - ) - - if item["type"] == "mcp_approval_response": - if approval_storage is not None: - try: - function_approval_request_content = await approval_storage.load_approval_request( - item["approval_request_id"] - ) - except (KeyError, ValueError) as exc: - raise _PublicResponseFailure(exc) from None - else: - raise ValueError("ApprovalStorage is required to load approval request.") - return Message( - role="user", - contents=[function_approval_request_content.to_function_approval_response(item["approve"])], - ) - - if item["type"] == "code_interpreter_call": - return Message( - role="assistant", - contents=[Content.from_code_interpreter_tool_call(call_id=item["id"])], - ) - - if item["type"] == "image_generation_call": - return Message( - role="assistant", - contents=[Content.from_image_generation_tool_call(image_id=item["id"])], - ) - - if item["type"] == "shell_call": - return Message( - role="assistant", - contents=[ - Content.from_shell_tool_call( - call_id=item["call_id"], - commands=item["action"]["commands"], - timeout_ms=item["action"].get("timeout_ms"), - max_output_length=item["action"].get("max_output_length"), - status=str(item.get("status")), - ) - ], - ) - - if item["type"] == "shell_call_output": - outputs = [_shell_command_output_to_content(out) for out in (item["output"] or [])] - return Message( - role="tool", - contents=[ - Content.from_shell_tool_result( - call_id=item["call_id"], - outputs=outputs, - max_output_length=item.get("max_output_length"), - ) - ], - ) - - if item["type"] == "local_shell_call": - commands = item["action"].get("command") or [] - return Message( - role="assistant", - contents=[ - Content.from_shell_tool_call( - call_id=item["call_id"], - commands=commands, - timeout_ms=item["action"].get("timeout_ms"), - status=str(item["status"]), - ) - ], - ) - - if item["type"] == "local_shell_call_output": - return Message( - role="tool", - contents=[ - Content.from_shell_tool_result( - call_id=item["id"], - outputs=[Content.from_shell_command_output(stdout=item["output"])], - ) - ], - ) - - if item["type"] == "file_search_call": - return Message( - role="assistant", - contents=[ - Content.from_function_call( - item["id"], - "file_search", - arguments=_json_safe_to_str({"queries": item["queries"]}), - informational_only=True, - hosted=True, - ) - ], - ) - - if item["type"] == "web_search_call": - return Message( - role="assistant", - contents=[Content.from_function_call(item["id"], "web_search", informational_only=True, hosted=True)], - ) - - if item["type"] == "computer_call": - plural_actions = item.get("actions") - singular_action = item.get("action") - actions = plural_actions or ([singular_action] if singular_action is not None else []) - if not actions: - raise ValueError("Computer call is missing its ordered actions.") - props = ( - { - "openai": {"computer_action_format": "single"}, - "computer_action_format": "single", - } - if not plural_actions and singular_action is not None - else None - ) - return Message( - role="assistant", - contents=[ - Content.from_computer_tool_call( - id=item["id"], - call_id=item["call_id"], - actions=actions, - status=item.get("status"), - pending_safety_checks=_computer_safety_checks(item.get("pending_safety_checks")), - additional_properties=props, - ) - ], - ) - - if item["type"] == "computer_call_output": - return Message( - role="tool", - contents=[ - Content.from_computer_tool_result( - id=item.get("id"), - call_id=item["call_id"], - screenshot=_computer_screenshot_from_output(item["output"]), - status=item.get("status"), - acknowledged_safety_checks=_computer_safety_checks(item.get("acknowledged_safety_checks")), - ) - ], - ) - - if item["type"] == "custom_tool_call": - return Message( - role="assistant", - contents=[ - Content.from_function_call( - item["call_id"], - item["name"], - arguments=item["input"], - informational_only=True, - hosted=True, - ) - ], - ) - - if item["type"] == "custom_tool_call_output": - output = _json_safe_to_str(item["output"]) - # Hosted-MCP results land here because the host writes them via - # `aoutput_item_custom_tool_call_output` (see `_OutputItemTracker.handle` for - # `mcp_server_tool_result`). The persisted `call_id` keeps its - # `mcp_*` prefix; on read, route those back to a hosted-MCP result - # Content so the chat-client serialize layer can coalesce them - # onto a single `mcp_call` input item with `output` populated. - # Issue #5546. - if item["call_id"] and item["call_id"].startswith("mcp_"): - return Message( - role="tool", - contents=[Content.from_mcp_server_tool_result(call_id=item["call_id"], output=output)], - ) - return Message( - role="tool", - contents=[Content.from_function_result(item["call_id"], result=output)], - ) - - if item["type"] == "apply_patch_call": - return Message( - role="assistant", - contents=[ - Content.from_function_call( - item["call_id"], - "apply_patch", - arguments=_json_safe_to_str(item["operation"]), - informational_only=True, - hosted=True, - ) - ], - ) - - if item["type"] == "apply_patch_call_output": - return Message( - role="tool", - contents=[Content.from_function_result(item["call_id"], result=_json_safe_to_str(item.get("output")))], - ) - - raise ValueError(f"Unsupported {_item_type_name} type: {item['type']}") - - -async def _output_items_to_messages( - history: Sequence[OutputItem], - *, - approval_storage: FunctionApprovalStore | None = None, -) -> list[Message]: - """Converts a sequence of OutputItem objects to a list of Message objects. - - Args: - history (Sequence[OutputItem]): The sequence of OutputItem objects to convert. - approval_storage (ApprovalStorage | None, optional): The approval storage to use for - resolving MCP approval requests. Defaults to None. - - Returns: - list[Message]: The list of Message objects. - """ - messages: list[Message] = [] - for item in history: - messages.append(await _output_item_to_message(item, approval_storage=approval_storage)) - return messages - - -async def _output_item_to_message( - item: OutputItem, *, approval_storage: FunctionApprovalStore | None = None -) -> Message: - """Converts an OutputItem to a Message. - - Args: - item (OutputItem): The OutputItem to convert. - approval_storage (ApprovalStorage | None, optional): The approval storage to use for - resolving MCP approval requests. Defaults to None. - - Returns: - Message: The converted Message. - - Raises: - ValueError: If the OutputItem type is not supported. - """ - if item["type"] == "oauth_consent_request": - item_data = cast("Mapping[str, Any]", item) - server_label = item_data.get("server_label") - additional_properties = ( - { - "openai": {"server_label": server_label}, - "server_label": server_label, - } - if isinstance(server_label, str) and server_label - else None - ) - return Message( - role="assistant", - contents=[ - Content.from_oauth_consent_request( - item["consent_link"], - additional_properties=additional_properties, - ) - ], - ) - - if item["type"] == "structured_outputs": - return Message(role="assistant", contents=[Content.from_text(_json_safe_to_str(item["output"]))]) - - return await _item_to_message( - cast(Item, item), - approval_storage=approval_storage, - _item_type_name="OutputItem", - ) - - -def _convert_output_message_content(content: OutputMessageContent) -> Content: - """Converts an OutputMessageContent to a Content object. - - Args: - content (OutputMessageContent): The OutputMessageContent to convert. - - Returns: - Content: The converted Content object. - - Raises: - ValueError: If the OutputMessageContent type is not supported. - """ - if content["type"] == "output_text": - return Content.from_text(content["text"]) - if content["type"] == "refusal": - return Content.from_text( - content["refusal"], - additional_properties=_refusal_additional_properties(), - ) - - # Defensive: `OutputMessageContent` currently only supports `output_text` and `refusal`, - # but if new types are added in the future, this will catch them. - raise ValueError(f"Unsupported OutputMessageContent type: {content['type']}") - - -def _convert_file_data(data_uri: str, filename: str | None = None) -> Content: - """Convert a file_data data URI to a Content object. - - For text/* MIME types, decodes the base64 content and returns it as text. - For other types, returns a URI-based Content with the filename preserved. - """ - # Parse data URI: data:;base64, - if data_uri.startswith("data:") and ";base64," in data_uri: - header, encoded = data_uri.split(";base64,", 1) - media_type = header[len("data:") :] - if media_type.startswith("text/"): - try: - decoded_text = base64.b64decode(encoded).decode("utf-8") - except (ValueError, UnicodeDecodeError): - logger.warning( - "Failed to decode text/* file_data as UTF-8, falling through to URI passthrough.", - exc_info=True, - ) - else: - prefix = f"[File: {filename}]\n" if filename else "" - return Content.from_text(f"{prefix}{decoded_text}") - additional_properties = ( - { - "openai": {"content_type": "input_file"}, - "openai_content_type": "input_file", - "filename": filename, - } - if filename - else { - "openai": {"content_type": "input_file"}, - "openai_content_type": "input_file", - } - ) - return Content.from_uri(data_uri, name=filename, additional_properties=additional_properties) - - -def _convert_message_content(content: MessageContent) -> Content: - """Converts a MessageContent to a Content object. - - Args: - content (MessageContent): The MessageContent to convert. - - Returns: - Content: The converted Content object. - - Raises: - ValueError: If the MessageContent type is not supported. - """ - if content["type"] == "input_text": - return Content.from_text(content["text"]) - if content["type"] == "output_text": - return Content.from_text(content["text"]) - if content["type"] == "text": - return Content.from_text(content["text"]) - if content["type"] == "summary_text": - return Content.from_text(content["text"]) - if content["type"] == "refusal": - return Content.from_text( - content["refusal"], - additional_properties=_refusal_additional_properties(), - ) - if content["type"] == "reasoning_text": - return Content.from_text_reasoning(text=content["text"]) - if content["type"] == "input_image": - detail = content.get("detail") - additional_properties = { - "openai": { - "content_type": "input_image", - **({"image_detail": detail} if isinstance(detail, str) else {}), - }, - "openai_content_type": "input_image", - **({"detail": detail} if isinstance(detail, str) else {}), - } - if image_url := content.get("image_url"): - if image_url.startswith("data:"): - return Content.from_uri(image_url, additional_properties=additional_properties) - return Content.from_uri(image_url, media_type="image/*", additional_properties=additional_properties) - if file_id := content.get("file_id"): - return Content.from_hosted_file(file_id, additional_properties=additional_properties) - if content["type"] == "input_file": - filename = content.get("filename") - additional_properties = { - "openai": {"content_type": "input_file"}, - "openai_content_type": "input_file", - **({"filename": filename} if isinstance(filename, str) else {}), - } - if file_url := content.get("file_url"): - return Content.from_uri( - file_url, - name=filename if isinstance(filename, str) else None, - additional_properties=additional_properties, - ) - if file_id := content.get("file_id"): - return Content.from_hosted_file( - file_id, - name=filename if isinstance(filename, str) else None, - additional_properties=additional_properties, - ) - if file_data := content.get("file_data"): - return _convert_file_data(file_data, content.get("filename")) - if content["type"] == "computer_screenshot": - return _computer_screenshot_from_output(content) - - raise ValueError(f"Unsupported MessageContent type: {content['type']}") - - -# endregion - -# region Output Item Conversion - - -def _json_default(value: Any) -> Any: - if is_dataclass(value) and not isinstance(value, type): - return asdict(value) - to_dict = getattr(value, "to_dict", None) - if callable(to_dict): - return to_dict() - return str(value) - - -def _json_safe_to_str(value: Any | None) -> str: - """Convert an argument or result value to a JSON-safe string. - - Args: - value: The value to convert, which can be a string, JSON-like object, or None. - - Returns: - The value as a JSON string. - """ - if value is None: - return "" - if isinstance(value, str): - return value - try: - return json.dumps(value, default=_json_default) - except (TypeError, ValueError): - return json.dumps(str(value)) - - -def _image_generation_result(outputs: Any) -> str | None: - """Return the base64 image of an image generation tool result, as the Responses API expects. - - Only a non-empty, valid base64 string or the valid base64 payload of a base64 data URI qualifies. Other URIs - (https, file, blob), data URIs without the base64 marker and malformed payloads are not base64, so they give - None. - """ - if isinstance(outputs, str): - return outputs if _is_base64(outputs) else None - items: Sequence[Any] = [outputs] - if isinstance(outputs, Sequence) and not isinstance(outputs, (bytes, bytearray)): - items = cast(Sequence[Any], outputs) - for item in items: - if isinstance(item, Content) and item.uri and item.uri.startswith("data:"): - # RFC 2397: only the metadata before the first comma can carry the marker, as its last parameter. - metadata, _, payload = item.uri.partition(",") - if metadata.endswith(";base64") and _is_base64(payload): - return payload - return None - - -def _is_base64(value: str) -> bool: - """Check whether a string is non-empty, valid base64.""" - if not value: - return False - try: - base64.b64decode(value, validate=True) - except ValueError: - return False - return True - - -def _reasoning_encrypted_content(content: Content) -> str | None: - """Return the opaque reasoning payload used for stateless replay.""" - encrypted_content = content.protected_data or content.additional_properties.get("encrypted_content") - return encrypted_content if isinstance(encrypted_content, str) else None - - -def _reasoning_output_item( - *, - item_id: str, - summary_texts: Sequence[str], - encrypted_content: str | None, - status: Literal["in_progress", "completed"], -) -> OutputItemReasoningItem: - """Build a hosted reasoning item while retaining provider replay metadata.""" - return OutputItemReasoningItem({ - "type": "reasoning", - "id": item_id, - "summary": [{"type": "summary_text", "text": text} for text in summary_texts], - "encrypted_content": encrypted_content, - "status": status, - }) - - -def _mcp_mapping_text(output: Mapping[Any, Any]) -> str | None: - """Extract text only from a recognized MCP text-content mapping.""" - text = output.get("text") - if not isinstance(text, str): - return None - if output.get("type") == "text" or set(output) == {"text"}: - return text - return None - - -def _stringify_mcp_output(output: Any) -> str: - """Convert hosted MCP output payloads into the string shape expected by mcp_call.output.""" - if output is None: - return "" - if isinstance(output, str): - return output - if isinstance(output, Mapping): - mapping = cast(Mapping[Any, Any], output) - if (text := _mcp_mapping_text(mapping)) is not None: - return text - return _json_safe_to_str(mapping) - if isinstance(output, Sequence) and not isinstance(output, (str, bytes, bytearray)): - parts: list[str] = [] - entries = cast(Sequence[Any], output) - for entry in entries: - if isinstance(entry, str): - parts.append(entry) - continue - if isinstance(entry, Content) and entry.type == "text": - parts.append(entry.text or "") - continue - if isinstance(entry, Mapping) and (text := _mcp_mapping_text(cast(Mapping[Any, Any], entry))) is not None: - parts.append(text) - continue - return _json_safe_to_str(entries) - return "".join(parts) - return _json_safe_to_str(output) - - -# endregion diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py new file mode 100644 index 00000000000..df0f9b314f9 --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_agent.py @@ -0,0 +1,1527 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Host session-owned agents through the Responses protocol.""" + +from __future__ import annotations + +import asyncio +import inspect +import logging +from collections.abc import ( + AsyncGenerator, + AsyncIterable, + Awaitable, + Generator, + Mapping, + Sequence, +) +from contextlib import AbstractAsyncContextManager, AsyncExitStack, aclosing, suppress +from copy import copy +from dataclasses import dataclass +from typing import Any, Literal, cast + +from agent_framework import ( + AgentResponse, + AgentResponseUpdate, + AgentSession, + ChatOptions, + ContextProvider, + FinishReason, + HistoryProvider, + InMemoryHistoryProvider, + Message, + RawAgent, + ResponseStream, + SessionStore, + SupportsAgentRun, +) +from agent_framework._telemetry import mark_feature_used +from agent_framework._tools import _load_pending_approval_requests # pyright: ignore[reportPrivateUsage] +from agent_framework.exceptions import AgentFrameworkException +from azure.ai.agentserver.core import get_request_context +from azure.ai.agentserver.responses import ( + ResponseContext, + ResponseProviderProtocol, + ResponsesServerOptions, +) +from azure.ai.agentserver.responses._id_generator import IdGenerator +from azure.ai.agentserver.responses.aio import ResponseEventStream +from azure.ai.agentserver.responses.hosting import ResponsesAgentServerHost +from azure.ai.agentserver.responses.models import ( + CreateResponse, + OAuthConsentRequestOutputItem, + ResponseStreamEvent, +) +from azure.ai.agentserver.responses.streaming._checkpoint import ResponseCheckpointEvent + +from ._feature_usage import FeatureIndex +from ._responses_shared import ( + _HOSTED_CONVERSATION_CLAIM_KEY, # pyright: ignore[reportPrivateUsage] + _HOSTED_CONVERSATION_COMMITTED_KEY, # pyright: ignore[reportPrivateUsage] + _HOSTED_PROVIDER_OUTPUT_COUNT_KEY, # pyright: ignore[reportPrivateUsage] + _HOSTED_PROVIDER_STATE_KEY, # pyright: ignore[reportPrivateUsage] + _HOSTED_PROVIDER_USAGE_KEY, # pyright: ignore[reportPrivateUsage] + _HOSTED_SERVICE_CHILD_KEY, # pyright: ignore[reportPrivateUsage] + _HOSTED_SOURCE_CONVERSATION_KEY, # pyright: ignore[reportPrivateUsage] + _PUBLIC_STREAM_FAILURE_MESSAGE, # pyright: ignore[reportPrivateUsage] + _agent_response_updates, # pyright: ignore[reportPrivateUsage] + _await_before_signal, # pyright: ignore[reportPrivateUsage] + _create_response_event_stream, # pyright: ignore[reportPrivateUsage] + _hosted_internal_metadata, # pyright: ignore[reportPrivateUsage] + _items_to_messages, # pyright: ignore[reportPrivateUsage] + _log_response_failure, # pyright: ignore[reportPrivateUsage] + _native_approval_id, # pyright: ignore[reportPrivateUsage] + _output_items_to_messages, # pyright: ignore[reportPrivateUsage] + _OutputItemTracker, # pyright: ignore[reportPrivateUsage] + _PublicResponseFailure, # pyright: ignore[reportPrivateUsage] + _reject_busy_conversation, # pyright: ignore[reportPrivateUsage] + _set_agent_session, # pyright: ignore[reportPrivateUsage] + _SignalledIterator, # pyright: ignore[reportPrivateUsage] +) +from ._scope import FoundryRequestScope +from ._shared import ( + AgentSessionStoreProvider, + AgentSource, + OptionsHook, + ResponsesRequest, + StoreProvider, + UnsupportedOptions, + _build_response_server_options, # pyright: ignore[reportPrivateUsage] + is_agent, + prepare_response_options, + resolve_agent, + response_run_options, + validate_agent_source, + validate_default_transport_options, + validate_request_options, + validate_unsupported_options, +) +from ._toolbox import ( + _is_allowed_oauth_consent_link, # pyright: ignore[reportPrivateUsage] + _normalize_allowed_oauth_consent_origins, # pyright: ignore[reportPrivateUsage] + consent_url_from_error, +) + +logger = logging.getLogger(__name__) + + +_HOSTED_RESPONSES_HISTORY_SOURCE_ID = "_foundry_responses_history" +_MAX_DEFERRED_APPROVAL_UPDATES = 4096 +_MAX_DEFERRED_APPROVAL_BYTES = 4 * 1024 * 1024 + +_HistorySource = Literal["host_managed", "agent_managed", "inference_service"] + + +def _is_hosted_responses_history_sentinel(provider: ContextProvider) -> bool: + """Return whether ``provider`` is the host's transient history buffer.""" + return ( + isinstance(provider, InMemoryHistoryProvider) + and provider.source_id == _HOSTED_RESPONSES_HISTORY_SOURCE_ID + and provider.load_messages + and provider.store_inputs + and not provider.store_context_messages + and provider.store_outputs + ) + + +@dataclass(frozen=True) +class _AgentConfiguration: + agent_server_history: bool + client_stores_by_default: bool + hosted_history: bool + + +def _validate_agent_configuration( + agent: SupportsAgentRun, + history_source: _HistorySource, + options: ResponsesServerOptions | None, + *, + background_source: Literal["host_managed", "inference_service"] = "host_managed", +) -> _AgentConfiguration: + resilient_background = bool(options and options.resilient_background) + if resilient_background and background_source != "inference_service": + raise RuntimeError( + "resilient_background=True requires background_source='inference_service' for regular agents. " + "Use ResponsesWorkflowHostServer for checkpoint-based workflow recovery." + ) + if background_source == "inference_service" and ( + not isinstance(agent, RawAgent) or getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", None) is not True + ): + raise RuntimeError("Provider background requires a RawAgent with a storing, resumable Responses client.") + if isinstance(agent, RawAgent): + identity_defaults = ("session_id", "agent_session_id", "user_id", "call_id", "service_session_id") + if conflicting := [name for name in identity_defaults if agent.default_options.get(name) is not None]: + raise RuntimeError(f"Model defaults cannot supply Foundry platform identity: {', '.join(conflicting)}.") + + uses_agent_server_history = history_source == "host_managed" + client_stores_by_default = False + if history_source != "agent_managed": + if not isinstance(agent, RawAgent): + raise RuntimeError( + "history_source='host_managed' and 'inference_service' require a RawAgent so hosting can enforce " + "downstream storage. Use history_source='agent_managed' for a custom SupportsAgentRun implementation " + "on stored requests." + ) + + stores_by_default = getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", None) + if not isinstance(stores_by_default, bool): + raise RuntimeError( + "history_source='host_managed' or 'inference_service' requires the chat client to declare " + "STORES_BY_DEFAULT so hosting can enforce downstream storage behavior." + ) + client_stores_by_default = stores_by_default + + if history_source == "inference_service" and not stores_by_default: + raise RuntimeError( + "history_source='inference_service' requires a storing client that declares STORES_BY_DEFAULT=True." + ) + + service_continuation_options = [ + name + for name in ("conversation_id", "previous_response_id", "conversation", "continuation_token", "response_id") + if agent.default_options.get(name) is not None + ] + if service_continuation_options: + raise RuntimeError( + f"history_source='host_managed' or 'inference_service' cannot use developer defaults for downstream " + f"continuation: {', '.join(service_continuation_options)}. " + "Use history_source='agent_managed' to retain developer-controlled continuation." + ) + + if history_source in ("host_managed", "inference_service"): + for provider in agent.context_providers: + if isinstance(provider, HistoryProvider) and provider.load_messages: + if history_source == "host_managed" and _is_hosted_responses_history_sentinel(provider): + continue + raise RuntimeError( + "The selected history_source conflicts with a load-enabled HistoryProvider. " + "Remove that provider or select history_source='agent_managed'." + ) + if history_source == "host_managed" and not stores_by_default and agent.default_options.get("store") is True: + raise RuntimeError( + "The chat client does not store by default, but the agent sets a downstream store option. " + "Remove that developer-owned default rather than letting hosting change it." + ) + + return _AgentConfiguration( + agent_server_history=uses_agent_server_history, + client_stores_by_default=client_stores_by_default, + hosted_history=uses_agent_server_history, + ) + + +def _initialize_agent_history(agent: SupportsAgentRun, configuration: _AgentConfiguration) -> None: + if not configuration.hosted_history or not isinstance(agent, RawAgent): + return + if not any( + _is_hosted_responses_history_sentinel(provider) + for provider in cast(Sequence[ContextProvider], agent.context_providers) + ): + agent.context_providers.append(InMemoryHistoryProvider(source_id=_HOSTED_RESPONSES_HISTORY_SOURCE_ID)) + + +class ResponsesHostServer(ResponsesAgentServerHost): + """Host an agent through the Responses protocol.""" + + def __init__( + self, + agent: AgentSource[ResponsesRequest], + *, + prefix: str = "", + response_store: ResponseProviderProtocol | None = None, + agent_session_store_provider: StoreProvider[SessionStore] | None = None, + allowed_oauth_consent_origins: Sequence[str] | None = None, + history_source: Literal["host_managed", "agent_managed", "inference_service"] = "host_managed", + background_source: Literal["host_managed", "inference_service"] = "host_managed", + max_history_items: int | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + resilient_background: bool = False, + prepare_options: OptionsHook | None = None, + unsupported_options: UnsupportedOptions = "warn", + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, + ) -> None: + """Initialize a ResponsesHostServer. + + Args: + agent: The agent to handle responses for, or a sync/async callable that creates one for each request. + Factories must declare a required parameter named `request`, accepting a `ResponsesRequest` + passed by keyword. Extra parameters must have defaults. The factory may ignore the request. + Factories run before `prepare_options`; the request exposes trusted scope, + this turn's input, and translated caller options before that hook. Use a callable for agents + that keep mutable state outside `AgentSession`. + `WorkflowAgent` instances and factory results are rejected. Use `ResponsesWorkflowHostServer` + with a native workflow factory instead. + Approval-emitting runs must finalize matching native pending session state; + hosting rejects unmatched requests before publishing them. + + Keyword Args: + prefix: The URL prefix for the server. + response_store: Optional response store for caller-facing persistence and history. + agent_session_store_provider: Optional provider for MAF agent session storage. + Stores the complete native session, including pending approvals, before an + approval request is published. If omitted, uses `AgentSessionStoreProvider`. + allowed_oauth_consent_origins: Optional exact HTTPS origins allowed for OAuth consent links. + When omitted, hosting retains its existing safe-HTTPS validation without restricting the + destination origin. When provided, every link must match an entry; an empty sequence rejects + every link. Entries must be origins such as `"https://auth.example.com"` and must not include + a path, query, or fragment. + history_source: Who supplies prior messages to the *model*, independently of the caller's + Responses `store` flag. Defaults to `"host_managed"`: + + - `"host_managed"`: Send the prior outer Responses transcript followed by this request's input. + For a storing chat client, force its per-run `store=False` and clear any restored + `service_session_id` to avoid replaying that transcript twice. A load-enabled + `HistoryProvider` or fixed downstream continuation default conflicts with this mode. + - `"inference_service"`: Send only this request's input. On caller `store=True`, run the inner + client with `store=True` and save its issued `service_session_id` in the host's + private MAF session store. The next stored request resumes that provider thread; + it must not branch an already-used provider conversation. + - `"agent_managed"`: Send only this request's input and preserve the agent's developer-owned + storage choice on stored requests. With an `InMemoryHistoryProvider` and + `default_options={"store": False}`, the provider loads prior messages from + the host-persisted MAF `AgentSession`. With `default_options={"store": True}`, + the downstream service retains history instead. + + For example, after a stored response to "My name is Ada", a stored follow-up + "What is my name?" sends both turns to the model with `"host_managed"`, but + sends only the follow-up with `"inference_service"` (plus a private service continuation). + With `"agent_managed"`, the developer chooses either of those agent-owned history sources. + background_source: `"host_managed"` (default) runs background work in the Responses host + without invoking provider-native background APIs; `"inference_service"` opts a storing, resumable + client into provider background polling and requires `history_source="inference_service"`. + Each poll retains the caller's run options and `background=True`; local tools must be + idempotent because a crash before the next private token is saved can replay them. + max_history_items: Positive cap on prior outer history items, only with host-managed history. + `None` leaves the installed SDK default unchanged; it does not mean unlimited. + The cap does not limit tokens, turns, retention, or workflow checkpoints. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + shutdown_grace_period_seconds: Positive shutdown wait in seconds, defaulting to 10. + The SDK also uses it when DELETE waits for response persistence. Not a request timeout. + resilient_background: Permit SDK recovery of eligible outer background responses, + defaulting to `False`. The SDK's `resilient_background` setting is provider-independent. + This regular-agent adapter currently recovers only saved inference-service continuations, + so it requires inference-service history and background. This does not start background + execution or guarantee exactly-once effects. + prepare_options: Developer hook to remove or replace caller model options for an agent. + unsupported_options: `"warn"` (default), `"error"`, or `"ignore"` when a custom agent + cannot accept runtime model options. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. + + Note: + The *caller* controls outer persistence with `POST /responses` `store=True/False`. + `store=False` writes no host-managed session or approval state and forces supported + inner clients not to store, regardless of `history_source`. The constructor's + `response_store=` argument selects the outer response-store backend. + The former constructor `store=` alias is rejected. Outer `response.id` is the polling/continuation handle; + any inner `service_session_id` is private and never replaces it. + + 1. With `history_source="host_managed"`, + the agent must not have a load-enabled history provider: the host supplies the transcript. + 2. Context providers must not keep required state only on their Python instances, + because the hosting environment may get deactivated between requests. Provider + state carried by `AgentSession`, including `InMemoryHistoryProvider` messages in + `history_source="agent_managed"` mode, is persisted by the configured session store. + 3. The server owns the supplied agent instance and may add hosting-specific providers. + Do not reuse the same agent with another host or invoke it directly after construction. + An agent returned by a callable belongs to that request. + 4. `resilient_background=True` supports regular agents configured with + `history_source="inference_service", background_source="inference_service"`. + For provider background, only a saved private continuation token can be polled after a crash; + a crash before the token is saved cannot be replayed safely. + Other regular-agent runs are not crash-recoverable. + 5. Steering is unavailable. Framework-generated SDK options keep steering disabled. + 6. A stored, named `"inference_service"` or `"agent_managed"` conversation is claimed in the scoped + session store before the agent runs. Conditional writes prevent a concurrent turn from mutating the same + provider thread. If the turn fails or is cancelled, start a new conversation instead of + reusing a potentially changed downstream thread. + + Raises: + TypeError: If the agent is a `WorkflowAgent`, or native workflow settings are supplied. + ValueError: If a host setting or its history/background applicability is invalid. + RuntimeError: If the agent configuration conflicts with the selected history source, + or the agent does not support the selected inference-service background mode. + """ + if history_source not in ("host_managed", "agent_managed", "inference_service"): + raise ValueError("history_source must be 'host_managed', 'agent_managed', or 'inference_service'.") + if background_source not in ("host_managed", "inference_service"): + raise ValueError("background_source must be 'host_managed' or 'inference_service'.") + if background_source == "inference_service" and history_source != "inference_service": + raise ValueError("Provider background requires history_source='inference_service'.") + if max_history_items is not None and history_source != "host_managed": + raise ValueError("max_history_items requires history_source='host_managed'.") + if resilient_background and background_source != "inference_service": + raise ValueError( + "resilient_background=True requires history_source='inference_service' and " + "background_source='inference_service' for regular agents." + ) + validate_agent_source(agent) + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + unsupported_options = validate_unsupported_options(unsupported_options) + options = _build_response_server_options( + max_history_items=max_history_items, + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=resilient_background, + ) + + resolved_agent = agent if is_agent(agent) else None + configuration = ( + _validate_agent_configuration(resolved_agent, history_source, options, background_source=background_source) + if resolved_agent is not None + else None + ) + + # No caller-owned agent state is mutated until all validation and base-host construction succeed. + self._allowed_oauth_consent_origins = _normalize_allowed_oauth_consent_origins(allowed_oauth_consent_origins) + super().__init__( + prefix=prefix, + options=options, + store=response_store, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) + + self._agent_source = agent + self._agent = resolved_agent + self._configuration = configuration + self._history_source: _HistorySource = history_source + self._background_source: Literal["host_managed", "inference_service"] = background_source + self._prepare_options = prepare_options + self._unsupported_options = unsupported_options + self._host_options = options + self._uses_agent_server_history = ( + configuration.agent_server_history if configuration is not None else history_source == "host_managed" + ) + self._resilient_background = resilient_background + if resolved_agent is not None and configuration is not None: + _initialize_agent_history(resolved_agent, configuration) + + # Storage providers + self._session_storage_provider = ( + AgentSessionStoreProvider() if agent_session_store_provider is None else agent_session_store_provider + ) + + # Lazy agent lifecycle: the agent (and any MCP tools it owns) is entered on + # the first request rather than at server startup, so that authentication + # failures during MCP connect can be surfaced to the client as an + # `oauth_consent_request` stream event instead of crashing the server. + self._agent_stack: AsyncExitStack | None = None + self._agent_init_lock = asyncio.Lock() + + self.shutdown_handler(self._cleanup_agent) + self.response_handler(self._handle_response) + + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + async def _ensure_agent_ready(self) -> None: + """Lazily enter the agent's async context exactly once. + + On failure the partial exit stack is closed and ``_agent_stack`` is left + as ``None`` so a subsequent request (e.g. after the user completes OAuth + consent) can retry the connection. + """ + if self._agent_stack is not None: + return + async with self._agent_init_lock: + if self._agent_stack is not None: + return + agent = self._agent + if agent is None: + raise RuntimeError("A request-scoped agent cannot use the server-lifetime initialization path.") + stack = AsyncExitStack() + try: + if isinstance(agent, AbstractAsyncContextManager): + await stack.enter_async_context(cast(AbstractAsyncContextManager[Any], agent)) + except BaseException: + await stack.aclose() + raise + self._agent_stack = stack + + async def _cleanup_agent(self) -> None: + """Close the agent's async context. Registered as the server shutdown handler.""" + stack = self._agent_stack + if stack is not None: + self._agent_stack = None + await stack.aclose() + + async def _handle_response( + self, + request: CreateResponse, + context: ResponseContext, + cancellation_signal: asyncio.Event, + ) -> AsyncIterable[ResponseStreamEvent | ResponseCheckpointEvent]: + """Handle the creation of a response.""" + response_event_stream = _create_response_event_stream(context, request) + if context.is_steered_turn: + logger.debug("Serving steered turn (pending_input_count=%d)", context.pending_input_count) + yield response_event_stream.emit_created() + yield response_event_stream.emit_in_progress() + + terminal_event: ResponseStreamEvent | None = None + try: + try: + scope = FoundryRequestScope.from_context( + self.config, get_request_context(), local_session_id=context.response_id + ) + except (TypeError, ValueError, RuntimeError) as exc: + raise _PublicResponseFailure(exc) from None + try: + hosted_request = ResponsesRequest(request, context, scope, response_run_options(request)) + except (TypeError, ValueError) as exc: + raise _PublicResponseFailure(exc) from None + agent = await resolve_agent(self._agent_source, hosted_request) + try: + configuration = self._configuration or _validate_agent_configuration( + agent, self._history_source, self._host_options, background_source=self._background_source + ) + except (TypeError, ValueError, RuntimeError) as exc: + raise _PublicResponseFailure(exc) from None + if self._configuration is None: + _initialize_agent_history(agent, configuration) + await prepare_response_options(hosted_request, self._prepare_options) + try: + validate_request_options(hosted_request.options) + except (TypeError, ValueError) as exc: + raise _PublicResponseFailure(exc) from None + except Exception as exc: + for event in self._emit_failure( + context.response_id, + "request preparation", + response_event_stream, + None, + exc, + ): + yield event + return + + async with AsyncExitStack() as resources: + inner = self._handle_prepared_response( + request, + context, + cancellation_signal, + response_event_stream, + agent, + configuration, + resources, + hosted_request, + ) + try: + async for event in inner: + if isinstance(event, Mapping) and event.get("type") in ( + "response.completed", + "response.incomplete", + "response.failed", + ): + terminal_event = event + else: + yield event + finally: + await inner.aclose() + if terminal_event is not None: + yield terminal_event + + async def _handle_prepared_response( + self, + request: CreateResponse, + context: ResponseContext, + cancellation_signal: asyncio.Event, + response_event_stream: ResponseEventStream, + agent: SupportsAgentRun, + configuration: _AgentConfiguration, + resources: AsyncExitStack, + hosted_request: ResponsesRequest, + ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: + # Lazy-enter the agent (and any MCP tools it owns). The MCP client wraps gateway + # consent failures (and other connection-time errors) in AgentFrameworkException; if + # one of those is a consent error we surface the consent link to the client through + # the already-opened response stream instead of failing the request. Other exception + # types fall through to the outer handler below and become ``response.failed``. + try: + if self._configuration is not None: + await self._ensure_agent_ready() + elif isinstance(agent, AbstractAsyncContextManager): + await resources.enter_async_context(agent) + except AgentFrameworkException as ex: + consent_errors_to_emit = consent_url_from_error(ex) + if consent_errors_to_emit is None or len(consent_errors_to_emit) == 0: + for event in self._emit_failure( + context.response_id, + "agent initialization", + response_event_stream, + None, + ex, + ): + yield event + return + + invalid_consent = next( + ( + consent_error + for consent_error in consent_errors_to_emit + if not _is_allowed_oauth_consent_link( + consent_error.consent_url, self._allowed_oauth_consent_origins + ) + ), + None, + ) + if invalid_consent is not None: + validation_error = ValueError( + f"OAuth consent request for tool '{invalid_consent.name}' must include an allowed safe HTTPS " + "consent link." + ) + for event in self._emit_failure( + context.response_id, + "OAuth consent validation", + response_event_stream, + None, + _PublicResponseFailure(validation_error), + ): + yield event + return + + if request.get("store") is False: + validation_error = ValueError("OAuth consent continuation requires store=true.") + for event in self._emit_failure( + context.response_id, + "OAuth consent validation", + response_event_stream, + None, + _PublicResponseFailure(validation_error), + ): + yield event + return + + try: + request_context = get_request_context() + session_storage = self._session_storage_provider.get_store( + config=self.config, platform_context=request_context + ) + previous_response_id = request.get("previous_response_id") + session_load_id = context.conversation_id or previous_response_id + session = await session_storage.get(session_load_id) if session_load_id is not None else None + if session is None: + if previous_response_id is not None and context.conversation_id is None: + raise _PublicResponseFailure( + RuntimeError( + "Cannot find an existing agent session for " + f"previous_response_id={previous_response_id}." + ) + ) + session = agent.create_session() + if context.conversation_id is not None: + _reject_busy_conversation(session) + if previous_response_id is not None and context.conversation_id is None: + if session.service_session_id is not None and session.state.get(_HOSTED_SOURCE_CONVERSATION_KEY): + raise _PublicResponseFailure( + ValueError("A service-managed downstream conversation cannot be forked.") + ) + session.state.pop(_HOSTED_SOURCE_CONVERSATION_KEY, None) + if context.conversation_id is not None: + session.state[_HOSTED_SOURCE_CONVERSATION_KEY] = context.conversation_id + await _set_agent_session(session_storage, context.response_id, session) + if context.conversation_id is not None: + await _set_agent_session(session_storage, context.conversation_id, session) + except Exception as save_error: + for event in self._emit_failure( + context.response_id, + "OAuth consent session persistence", + response_event_stream, + None, + save_error, + ): + yield event + return + + for consent_error in consent_errors_to_emit: + logger.warning("Consent URL for tool '%s': %s", consent_error.name, consent_error.consent_url) + oauth_item = OAuthConsentRequestOutputItem( + id=IdGenerator.new_id("oacr"), + response_id=context.response_id, + type="oauth_consent_request", + consent_link=consent_error.consent_url, + server_label=consent_error.name, + ) + builder = response_event_stream.add_output_item(oauth_item["id"]) + yield builder.emit_added(oauth_item) + yield builder.emit_done(oauth_item) + + yield response_event_stream.emit_incomplete() + return + + tracker = _OutputItemTracker(response_event_stream, self._allowed_oauth_consent_origins) + try: + inner = self._handle_inner_agent( + request, + context, + response_event_stream, + tracker, + cancellation_signal, + agent, + configuration, + hosted_request, + ) + + try: + async for event in inner: + yield event + except BaseException: + await inner.aclose() + raise + + if cancellation_signal.is_set() and context.client_cancelled: + # A cancelled run drains the inner generator without raising + # (``_handle_inner_agent`` stops its + # ``_SignalledIterator`` loop and return normally once the signal fires). + # Emit nothing here so a caller cannot mistake this for a normal + # completion; the host server's cancel-aware layer synthesizes the + # cancelled terminal when the handler returns without one. Gated on + # ``client_cancelled`` (not just the signal) because steering pressure + # also sets ``cancellation_signal`` without that cause flag; a steered + # turn must still drain ``tracker.close()`` and emit its normal terminal + # below so its partial output is not misreported as a failure. + return + + for event in tracker.close(): + yield event + + if cancellation_signal.is_set() and context.client_cancelled: + # Draining ``tracker.close()`` yields events one at a time, and each + # ``yield`` above suspends this handler until the caller resumes it. + # A cancellation can arrive during that window, after the earlier check + # already passed, so it must be rechecked here, immediately before + # selecting the terminal event. Same ``client_cancelled`` gate as above. + return + + tracker.discard_pending_container_file_citations() + incomplete_reason = tracker.incomplete_reason + if tracker.oauth_consent_requested or incomplete_reason is not None: + yield response_event_stream.emit_incomplete(reason=incomplete_reason, usage=tracker.usage) + else: + yield response_event_stream.emit_completed(usage=tracker.usage) + except Exception as ex: + for event in tracker.close(): + yield event + + for event in self._emit_failure( + context.response_id, + "response production", + response_event_stream, + tracker, + ex, + ): + yield event + + async def _load_request_messages( + self, + context: ResponseContext, + *, + session: AgentSession | Awaitable[AgentSession] | None = None, + configuration: _AgentConfiguration | None = None, + ) -> list[Message]: + """Load the request's input and prior history concurrently, assembled for the run. + + The caller's input items and the conversation history are independent + storage round-trips with no data dependency, so they are fetched in + parallel to remove serial latency from the request critical path. The + history read is only issued when AgentServer is the history source; in + stateless single-turn requests it short-circuits without a round-trip. + + Returns the messages already ordered as model input (history precedes + input), so the message-ordering rule lives only here and callers do not + need to know the storage-result ordering. If either read fails, the + sibling task is cancelled and drained so no storage read is orphaned. + """ + uses_agent_server_history = ( + configuration.agent_server_history if configuration is not None else self._uses_agent_server_history + ) + + async def _load_input() -> list[Message]: + input_items = await context.get_input_items() + resolved_session = await session if inspect.isawaitable(session) else session + wire_ids: list[str] = [] + for input_item in input_items: + if input_item["type"] == "mcp_approval_response": + wire_ids.append(input_item["approval_request_id"]) + elif input_item["type"] == "mcp_approval_request": + wire_ids.append(input_item["id"]) + approval_ids: dict[str, str] = {} + if wire_ids: + # The SDK context has no public exact-item lookup. Use its already selected provider, + # not capped model history or a second approval store, preserving the platform scope. + provider = context._provider # pyright: ignore[reportPrivateUsage] + if provider is None or resolved_session is None: + raise _PublicResponseFailure(ValueError("Approval continuation requires a stored native session.")) + stored_items = await provider.get_items(wire_ids, context=context.platform_context) + for wire_id, item in zip(wire_ids, stored_items, strict=True): + if item is None or item["type"] != "mcp_approval_request" or item["id"] != wire_id: + raise _PublicResponseFailure(ValueError(f"No native pending approval matches {wire_id!r}.")) + approval_ids[wire_id] = _native_approval_id(cast(Mapping[str, Any], item), session=resolved_session) + return await _items_to_messages( + input_items, + session=resolved_session, + approval_ids=approval_ids, + include_hosted_approval_requests=uses_agent_server_history, + ) + + async def _load_history() -> list[Message]: + if not uses_agent_server_history: + return [] + history = await context.get_history() + return await _output_items_to_messages(history) + + input_task = asyncio.ensure_future(_load_input()) + history_task = asyncio.ensure_future(_load_history()) + try: + input_messages, history_messages = await asyncio.gather(input_task, history_task) + except BaseException: + # gather surfaces the first failure without cancelling the sibling, and a + # cancellation of this coroutine must not leave either read running. Cancel + # both and await them so no storage operation is orphaned after we unwind. + input_task.cancel() + history_task.cancel() + await asyncio.gather(input_task, history_task, return_exceptions=True) + raise + return [*history_messages, *input_messages] + + async def _handle_inner_agent( + self, + request: CreateResponse, + context: ResponseContext, + response_event_stream: ResponseEventStream, + tracker: _OutputItemTracker, + cancellation_signal: asyncio.Event, + agent: SupportsAgentRun, + configuration: _AgentConfiguration, + hosted_request: ResponsesRequest, + ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: + """Handle a regular (non-workflow) agent. + + The response stream, tracker, and opening lifecycle events are produced + by :meth:`_handle_response`, which also converts any raised exception + into a terminal ``response.failed`` event (draining the tracker so the + SSE stream stays well-formed). + """ + provider_background = self._background_source == "inference_service" and request.get("background") is True + if context.is_recovery and not provider_background: + raise RuntimeError("A non-resumable agent cannot be replayed after a process crash.") + + stored = request.get("store") is not False + + request_messages_task: asyncio.Task[list[Message]] | None = None + conversation_already_committed = False + try: + if isinstance(agent, RawAgent): + try: + validate_default_transport_options( + agent.default_options, + allow_agent_store=self._history_source == "agent_managed" and stored, + ) + except (TypeError, ValueError) as exc: + raise _PublicResponseFailure(exc) from None + if not stored: + if not isinstance(agent, RawAgent): + raise _PublicResponseFailure( + RuntimeError( + "store=false cannot guarantee that a custom agent will disable inner storage; " + "use a RawAgent or store=true." + ) + ) + continuation_defaults = [ + name + for name in ( + "conversation_id", + "previous_response_id", + "conversation", + "continuation_token", + "response_id", + ) + if agent.default_options.get(name) is not None + ] + if continuation_defaults: + raise RuntimeError( + "store=false cannot use developer defaults for downstream service continuation: " + + ", ".join(continuation_defaults) + ) + if any( + isinstance(provider, HistoryProvider) + and not isinstance(provider, InMemoryHistoryProvider) + and (provider.store_inputs or provider.store_context_messages or provider.store_outputs) + for provider in agent.context_providers + ): + raise _PublicResponseFailure( + RuntimeError( + "store=false cannot prevent an external HistoryProvider from persisting responses. " + "Use an InMemoryHistoryProvider or store=true." + ) + ) + if self._history_source == "agent_managed": + stores_by_default = getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", None) + if not isinstance(stores_by_default, bool): + raise RuntimeError( + "store=false with history_source='agent_managed' requires a client declaring " + "STORES_BY_DEFAULT; " + "select history_source='host_managed' or 'inference_service', or use store=true." + ) + if not stores_by_default and agent.default_options.get("store") is True: + raise RuntimeError( + "store=false with history_source='agent_managed' cannot safely override a storing " + "default on a client that does not declare storage support." + ) + + request_context = get_request_context() + previous_response_id = request.get("previous_response_id") + session_load_id = ( + context.response_id + if context.is_recovery and provider_background + else context.conversation_id or previous_response_id + ) + if not stored and session_load_id is not None and self._history_source == "inference_service": + raise ValueError( + "store=false cannot resume service-managed history; start a new one-shot request " + "or use history_source='host_managed' or 'agent_managed'." + ) + session_storage = ( + self._session_storage_provider.get_store(config=self.config, platform_context=request_context) + if stored or (session_load_id is not None and self._history_source == "agent_managed") + else None + ) + + session_ready: asyncio.Future[AgentSession] = asyncio.get_running_loop().create_future() + request_messages_task = asyncio.ensure_future( + self._load_request_messages(context, session=session_ready, configuration=configuration) + ) + session = ( + await session_storage.get(session_load_id) + if session_storage is not None and session_load_id is not None + else None + ) + if session is None: + if context.is_recovery and provider_background: + raise _PublicResponseFailure( + RuntimeError("Provider background was interrupted before its continuation token was stored.") + ) + if stored and previous_response_id is not None and context.conversation_id is None: + raise _PublicResponseFailure( + RuntimeError( + f"Cannot find an existing agent session for previous_response_id={previous_response_id}." + ) + ) + session = agent.create_session() + session_ready.set_result(session) + provider_state = session.state.get(_HOSTED_PROVIDER_STATE_KEY) + if context.conversation_id is not None: + if context.is_recovery and provider_background: + if session_storage is None: + raise RuntimeError("Provider background recovery requires agent session storage.") + head = await session_storage.get(context.conversation_id) + conversation_already_committed = ( + head is not None + and head.state.get(_HOSTED_CONVERSATION_CLAIM_KEY) is None + and head.state.get(_HOSTED_CONVERSATION_COMMITTED_KEY) == context.response_id + and isinstance(provider_state, Mapping) + and cast(Mapping[str, Any], provider_state).get("completed") is True + ) + if not conversation_already_committed and ( + head is None or head.state.get(_HOSTED_CONVERSATION_CLAIM_KEY) != context.response_id + ): + raise _PublicResponseFailure( + RuntimeError( + "Cannot recover provider background: the service-backed conversation claim " + "is no longer held by this response." + ) + ) + else: + _reject_busy_conversation(session) + if not stored and self._history_source == "agent_managed" and session.service_session_id is not None: + raise _PublicResponseFailure( + ValueError( + "store=false cannot continue agent-managed downstream service history; " + "start a new one-shot request or use store=true." + ) + ) + if ( + not context.is_recovery + and provider_state is not None + and ( + not isinstance(provider_state, Mapping) + or cast(Mapping[str, Any], provider_state).get("completed") is not True + ) + ): + raise _PublicResponseFailure( + ValueError("A provider background response must complete before the next turn.") + ) + if previous_response_id is not None and context.conversation_id is None and not context.is_recovery: + if session.service_session_id is not None and session.state.get(_HOSTED_SOURCE_CONVERSATION_KEY): + raise _PublicResponseFailure( + ValueError("A service-managed downstream conversation cannot be forked.") + ) + if ( + stored + and self._history_source in ("inference_service", "agent_managed") + and session.service_session_id is not None + and session.state.get(_HOSTED_SERVICE_CHILD_KEY) + ): + raise _PublicResponseFailure(ValueError("A service-managed downstream response cannot be forked.")) + except BaseException as ex: + # Session preparation failed (or the request was cancelled / the stream closed — + # neither of which is an Exception). Cancel and drain the in-flight message-loading + # task so it is not orphaned, and log only ordinary failures. + if request_messages_task is not None: + request_messages_task.cancel() + with suppress(BaseException): + await request_messages_task + if isinstance(ex, Exception): + _log_response_failure("state storage preparation", context.response_id, ex) + raise + + request_failure: Exception | None = None + save_failure: Exception | None = None + request_interrupted = False + + try: + if configuration.agent_server_history: + session.state.pop(_HOSTED_RESPONSES_HISTORY_SOURCE_ID, None) + # A restored service ID belongs to the downstream model service. Replaying the + # AgentServer transcript while resuming that service history would duplicate every + # prior turn, so AgentServer-history mode always starts the model call statelessly. + session.service_session_id = None + + messages = await request_messages_task + run_kwargs: dict[str, Any] = { + "messages": messages, + "session": session, + } + chat_options = cast(ChatOptions[Any], dict(hosted_request.options)) + are_options_set = bool(chat_options) + if configuration.agent_server_history: + if configuration.client_stores_by_default: + # The response provider already owns the transcript used for this run. Keep a + # storing downstream service stateless so it cannot become a second history source. + chat_options["store"] = False + else: + # Do not pass a storage option to clients that do not advertise support for it. + chat_options.pop("store", None) + elif self._history_source == "inference_service": + chat_options["store"] = stored + if not stored: + session.service_session_id = None + elif ( + not stored + and isinstance(agent, RawAgent) + and ( + getattr(cast(Any, agent).client, "STORES_BY_DEFAULT", False) + or agent.default_options.get("store") is not None + ) + ): + chat_options["store"] = False + + if isinstance(agent, RawAgent): + run_kwargs["options"] = chat_options + elif are_options_set: + if self._unsupported_options == "error": + raise _PublicResponseFailure(TypeError("The hosted agent does not accept caller runtime options.")) + if self._unsupported_options == "warn": + logger.warning("Agent doesn't support runtime options. They will be ignored.") + + if previous_response_id is not None and context.conversation_id is None and not context.is_recovery: + if ( + stored + and self._history_source in ("inference_service", "agent_managed") + and session.service_session_id is not None + ): + if session_storage is None: + raise RuntimeError("Service history requires agent session storage.") + session.state[_HOSTED_SERVICE_CHILD_KEY] = context.response_id + try: + await _set_agent_session(session_storage, previous_response_id, session) + finally: + session.state.pop(_HOSTED_SERVICE_CHILD_KEY, None) + session.state.pop(_HOSTED_SOURCE_CONVERSATION_KEY, None) + + if not context.is_recovery: + session.state.pop(_HOSTED_PROVIDER_STATE_KEY, None) + session.state.pop(_HOSTED_CONVERSATION_COMMITTED_KEY, None) + + if ( + stored + and context.conversation_id is not None + and not configuration.agent_server_history + and not conversation_already_committed + ): + if session_storage is None: + raise RuntimeError("Service history requires agent session storage.") + # The scoped store's conditional write must succeed before the provider can + # mutate its linear thread; a losing turn never reaches agent.run(). + session.state[_HOSTED_CONVERSATION_CLAIM_KEY] = context.response_id + try: + await _set_agent_session(session_storage, context.conversation_id, session) + finally: + session.state.pop(_HOSTED_CONVERSATION_CLAIM_KEY, None) + + inner_stream: ResponseStream[AgentResponseUpdate, AgentResponse[Any]] | None = None + updates: _SignalledIterator[AgentResponseUpdate] | None = None + deferred_updates: list[AgentResponseUpdate] = [] + deferred_bytes = 0 + + async def publish_update( + update: AgentResponseUpdate, *, complete_message: bool = False + ) -> AsyncGenerator[ResponseStreamEvent]: + approval_requests = [ + content for content in update.contents if content.type == "function_approval_request" + ] + if approval_requests: + if not stored or session_storage is None: + raise ValueError("Approval and user-input continuation requires store=true.") + pending = _load_pending_approval_requests(session) + for request in approval_requests: + recorded = pending.get(request.id) if request.id is not None else None + if recorded is None or recorded.to_dict() != request.to_dict(): + raise _PublicResponseFailure( + ValueError( + "The agent emitted an approval request without matching native pending state. " + "Use native agent/session approval handling before the run completes." + ) + ) + await _set_agent_session(session_storage, context.response_id, session) + if context.conversation_id is not None: + await _set_agent_session(session_storage, context.conversation_id, session) + async for event in tracker.handle_update( + update, complete_message=complete_message, approval_session_id=session.session_id + ): + yield event + + if provider_background: + if session_storage is None or not isinstance(agent, RawAgent): + raise RuntimeError("Provider background requires a stored MAF agent session.") + output_count = _hosted_internal_metadata(response_event_stream).get( + _HOSTED_PROVIDER_OUTPUT_COUNT_KEY, 0 + ) + if type(output_count) is not int or output_count < 0: + raise RuntimeError("The persisted provider output cursor is invalid.") + responses = self._provider_background_responses( + agent=cast(RawAgent[ChatOptions[Any]], agent), + messages=messages, + session=session, + session_storage=session_storage, + options=chat_options, + context=context, + cancellation_signal=cancellation_signal, + emitted_output_count=output_count, + ) + async with aclosing(responses): + async for response, output_count in responses: + for update in _agent_response_updates(response, context.response_id): + if context.shutdown.is_set() or cancellation_signal.is_set(): + break + async for event in publish_update(update, complete_message=True): + yield event + if context.shutdown.is_set() or cancellation_signal.is_set(): + continue + for event in tracker.close(): + yield event + private_metadata = _hosted_internal_metadata(response_event_stream) + private_metadata[_HOSTED_PROVIDER_OUTPUT_COUNT_KEY] = output_count + private_metadata[_HOSTED_PROVIDER_USAGE_KEY] = tracker.usage_details + if self._resilient_background: + yield response_event_stream.checkpoint() + else: + inner_stream = agent.run(stream=True, **run_kwargs) # type: ignore[reportUnknownMemberType] + updates = _SignalledIterator(inner_stream, context.shutdown, cancellation_signal) + async with aclosing(updates): + async for update in updates: + if not stored and any( + content.type in ("function_approval_request", "oauth_consent_request") + or content.user_input_request + for content in update.contents + ): + if any(content.type == "oauth_consent_request" for content in update.contents): + raise _PublicResponseFailure( + ValueError("OAuth consent continuation requires store=true.") + ) + raise ValueError("Approval and user-input continuation requires store=true.") + if deferred_updates or any( + content.type == "function_approval_request" for content in update.contents + ): + # Native providers can record approval authority only after their stream + # finishes. Keep subsequent output ordered behind that approval. + if len(deferred_updates) >= _MAX_DEFERRED_APPROVAL_UPDATES: + raise _PublicResponseFailure( + ValueError("Deferred approval output exceeds the update-count limit.") + ) + update_bytes = len( + update.to_json(ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode( + "utf-8" + ) + ) + if deferred_bytes + update_bytes > _MAX_DEFERRED_APPROVAL_BYTES: + raise _PublicResponseFailure( + ValueError("Deferred approval output exceeds the serialized-size limit.") + ) + deferred_bytes += update_bytes + deferred_updates.append(update) + else: + async for event in publish_update(update): + yield event + if inner_stream is not None and isinstance(updates, _SignalledIterator) and not updates.signalled: + final = await inner_stream.get_final_response() + if final.continuation_token is not None and final.finish_reason is None: + raise RuntimeError( + "The inner agent returned an unfinished provider response; " + "configure background_source='inference_service' with history_source='inference_service' " + "to resume it." + ) + for update in deferred_updates: + async for event in publish_update(update): + yield event + except (asyncio.CancelledError, GeneratorExit): + request_interrupted = True + raise + except Exception as ex: + request_failure = ex + _log_response_failure("agent execution", context.response_id, ex) + finally: + if configuration.hosted_history: + session.state.pop(_HOSTED_RESPONSES_HISTORY_SOURCE_ID, None) + if not provider_background and not context.is_recovery: + session.state.pop(_HOSTED_PROVIDER_STATE_KEY, None) + + # Never persist a session that could resume an inner response contrary to the + # history mode or the caller's explicit storage decision. + stored_output_violation = ( + self._history_source == "host_managed" or not stored + ) and session.service_session_id is not None + if stored_output_violation: + misconfigured = RuntimeError( + "The agent's chat client stored this turn server-side despite store=False for the inner model. " + "Configure the client to honor store=False, or use history_source='inference_service' " + "with store=true." + ) + _log_response_failure("stored output validation", context.response_id, misconfigured) + if request_failure is None and not request_interrupted: + request_failure = _PublicResponseFailure(misconfigured) + if ( + self._history_source == "inference_service" + and stored + and _HOSTED_PROVIDER_STATE_KEY not in session.state + and session.service_session_id is None + and request_failure is None + and not request_interrupted + and not (cancellation_signal.is_set() and context.client_cancelled) + ): + request_failure = RuntimeError( + "history_source='inference_service' requires the chat client to return a service continuation ID." + ) + try: + provider_state = session.state.get(_HOSTED_PROVIDER_STATE_KEY) + final_provider_state = not provider_background or ( + request_failure is None + and not request_interrupted + and not (cancellation_signal.is_set() and context.client_cancelled) + and ( + provider_state is None + or ( + isinstance(provider_state, Mapping) + and cast(Mapping[str, Any], provider_state).get("completed") is True + ) + ) + ) + superseded_by_steering = bool(self._host_options and self._host_options.steerable_conversations) and ( + cancellation_signal.is_set() and not context.client_cancelled and not context.shutdown.is_set() + ) + if stored and session_storage is not None and not stored_output_violation and final_provider_state: + if context.conversation_id is not None: + session.state[_HOSTED_SOURCE_CONVERSATION_KEY] = context.conversation_id + await _set_agent_session(session_storage, context.response_id, session) + if ( + context.conversation_id is not None + and not conversation_already_committed + and not superseded_by_steering + and not request_interrupted + and request_failure is None + and ( + configuration.agent_server_history + or (not cancellation_signal.is_set() and not context.shutdown.is_set()) + ) + ): + if provider_background: + session.state.pop(_HOSTED_PROVIDER_STATE_KEY, None) + session.state[_HOSTED_CONVERSATION_COMMITTED_KEY] = context.response_id + await _set_agent_session(session_storage, context.conversation_id, session) + except Exception as save_error: + save_failure = save_error + if request_interrupted: + message = "Failed to persist the Agent Framework session while unwinding an interrupted request" + elif request_failure is not None: + message = "Failed to persist the Agent Framework session after an agent failure" + else: + message = "Failed to persist the Agent Framework session after a successful request" + _log_response_failure(message, context.response_id, save_error) + + if request_failure is not None and save_failure is not None: + raise RuntimeError( + f"Agent request failed: {str(request_failure) or type(request_failure).__name__}; " + f"session persistence also failed: {str(save_failure) or type(save_failure).__name__}" + ) + elif request_failure is not None: + raise request_failure + elif save_failure is not None: + raise save_failure + + async def _provider_background_responses( + self, + *, + agent: RawAgent[ChatOptions[Any]], + messages: list[Message], + session: AgentSession, + session_storage: SessionStore, + options: ChatOptions[Any], + context: ResponseContext, + cancellation_signal: asyncio.Event, + emitted_output_count: int, + ) -> AsyncGenerator[tuple[AgentResponse[Any], int]]: + """Persist polling output with its private token and replay uncheckpointed output on recovery.""" + outputs: list[dict[str, Any]] = [] + + async def proceed(*, has_token: bool) -> bool: + if cancellation_signal.is_set() and context.client_cancelled: + if not has_token: + logger.warning( + "Provider background submission was cancelled before a token was saved; " + "remote work may continue." + ) + return False + if context.shutdown.is_set(): + if has_token and self._resilient_background: + await context.exit_for_recovery() + if has_token: + raise RuntimeError("Provider background recovery requires resilient_background=True.") + raise _PublicResponseFailure( + RuntimeError( + "Provider background submission stopped before its token was saved; cannot safely retry it." + ) + ) + if cancellation_signal.is_set(): + raise RuntimeError("Provider background was interrupted without a client cancellation.") + return True + + async def run_provider( + options: ChatOptions[Any], *, input_messages: list[Message] | None, phase: Literal["submit", "poll"] + ) -> AgentResponse[Any]: + try: + return await agent.run(input_messages, session=session, options=options) + except Exception as exc: + logger.warning("Inner provider background %s failed (%s).", phase, type(exc).__name__) + raise RuntimeError(f"Inner provider background {phase} failed; inspect the host logs.") + + async def save_private_state() -> None: + try: + await session_storage.set(context.response_id, session) + except Exception as exc: + logger.warning("Private provider background state save failed (%s).", type(exc).__name__) + else: + return + raise RuntimeError("Could not save private provider background state; inspect the host logs.") + + def record_output(response: AgentResponse[Any]) -> None: + messages = response.messages + if response.continuation_token is not None: + # A tool loop prefixes completed calls/results to the unfinished model + # response. Retain that prefix, not partial output that polling will repeat. + for index in range(len(messages) - 1, -1, -1): + message = messages[index] + last_result = next( + ( + offset + for offset in range(len(message.contents) - 1, -1, -1) + if message.contents[offset].type == "function_result" + ), + None, + ) + if last_result is not None: + completed_message = copy(message) + completed_message.contents = message.contents[: last_result + 1] + messages = [*messages[:index], completed_message] + break + else: + return + outputs.append( + AgentResponse( + messages=messages, + usage_details=response.usage_details, + finish_reason=FinishReason(response.finish_reason) if response.finish_reason is not None else None, + ).to_dict() + ) + + async def pending_outputs() -> AsyncGenerator[tuple[AgentResponse[Any], int]]: + nonlocal emitted_output_count + while emitted_output_count < len(outputs): + if not await proceed(has_token=True): + return + yield AgentResponse.from_dict(outputs[emitted_output_count]), emitted_output_count + 1 + if not await proceed(has_token=True): + return + emitted_output_count += 1 + + if context.is_recovery: + saved = session.state.get(_HOSTED_PROVIDER_STATE_KEY) + if not isinstance(saved, Mapping): + raise _PublicResponseFailure( + RuntimeError("Cannot recover a provider background job without its stored continuation token.") + ) + saved_payload = cast(Mapping[str, Any], saved) + if saved_payload.get("outer_response_id") != context.response_id: + raise _PublicResponseFailure( + RuntimeError("Cannot recover a provider background job without its stored continuation token.") + ) + token = saved_payload.get("continuation_token") + if not isinstance(token, Mapping): + raise RuntimeError("The stored provider continuation token is invalid.") + continuation_token: Mapping[str, Any] = cast(Mapping[str, Any], token) + saved_outputs = saved_payload.get("outputs", []) + if not isinstance(saved_outputs, list) or any( + not isinstance(output, dict) for output in cast(list[object], saved_outputs) + ): + raise RuntimeError("The stored provider output is invalid.") + outputs = cast(list[dict[str, Any]], saved_outputs) + if emitted_output_count > len(outputs): + raise RuntimeError("The persisted provider output cursor exceeds its private snapshot.") + async for output in pending_outputs(): + yield output + if saved_payload.get("completed") is True and outputs: + await proceed(has_token=True) + return + else: + if not await proceed(has_token=False): + return + completed, first = await _await_before_signal( + lambda: run_provider( + cast(ChatOptions[Any], {**options, "background": True}), + input_messages=messages, + phase="submit", + ), + context.shutdown, + cancellation_signal, + ) + if not completed: + if not await proceed(has_token=False): + return + raise RuntimeError("Provider background submission was interrupted before its token was saved.") + if first is None: + raise RuntimeError("The provider did not return a background response.") + if first.continuation_token is None: + if not await proceed(has_token=False): + return + yield first, 1 + return + if not isinstance(first.continuation_token, Mapping): + raise RuntimeError("The provider returned a continuation token that cannot be persisted.") + continuation_token = cast(Mapping[str, Any], first.continuation_token) + if any(message.contents for message in first.messages) or first.usage_details: + record_output(first) + session.state[_HOSTED_PROVIDER_STATE_KEY] = { + "outer_response_id": context.response_id, + "continuation_token": dict(continuation_token), + "outputs": outputs, + } + await save_private_state() + async for output in pending_outputs(): + yield output + + while True: + if not await proceed(has_token=True): + return + slept, _ = await _await_before_signal(lambda: asyncio.sleep(2), context.shutdown, cancellation_signal) + if not slept: + if not await proceed(has_token=True): + return + raise RuntimeError("Provider background polling was interrupted.") + if not await proceed(has_token=True): + return + completed, current = await _await_before_signal( + lambda token=continuation_token: run_provider( + cast( + ChatOptions[Any], + {**options, "background": True, "store": True, "continuation_token": token}, + ), + input_messages=None, + phase="poll", + ), + context.shutdown, + cancellation_signal, + ) + if not completed: + if not await proceed(has_token=True): + return + raise RuntimeError("Provider background polling was interrupted.") + if current is None: + raise RuntimeError("The provider did not return a background response.") + if current.continuation_token is None: + # Save the completed output with its token before publishing it; recovery + # replays the uncheckpointed output without invoking the agent again. + record_output(current) + session.state[_HOSTED_PROVIDER_STATE_KEY] = { + "outer_response_id": context.response_id, + "continuation_token": dict(continuation_token), + "completed": True, + "outputs": outputs, + } + await save_private_state() + async for output in pending_outputs(): + yield output + return + if not isinstance(current.continuation_token, Mapping): + raise RuntimeError("The provider returned a continuation token that cannot be persisted.") + if current.continuation_token == continuation_token: + continue + if any(message.contents for message in current.messages) or current.usage_details: + record_output(current) + continuation_token = cast(Mapping[str, Any], current.continuation_token) + session.state[_HOSTED_PROVIDER_STATE_KEY] = { + "outer_response_id": context.response_id, + "continuation_token": dict(continuation_token), + "outputs": outputs, + } + await save_private_state() + async for output in pending_outputs(): + yield output + + @staticmethod + def _emit_failure( + response_id: str, + phase: str, + response_event_stream: ResponseEventStream, + tracker: _OutputItemTracker | None, + ex: BaseException, + *, + public_message: str | None = None, + ) -> Generator[ResponseStreamEvent]: + """Log ``ex`` and yield a terminal ``response.failed`` event. + + Drains any in-progress streaming output item first so the resulting + SSE stream stays well-formed, then emits ``response.failed`` carrying a + stable public message unless the caller explicitly supplies a safe + validation message. Any error raised while draining the tracker is + logged and otherwise ignored so that the original failure remains the + terminal failure. + """ + _log_response_failure(phase, response_id, ex) + if tracker is not None: + try: + yield from tracker.close() + except Exception as close_error: + _log_response_failure("failed tracker cleanup", response_id, close_error) + tracker.discard_pending_container_file_citations() + if public_message is not None: + message = public_message + elif isinstance(ex, _PublicResponseFailure): + message = ex.public_message + else: + message = _PUBLIC_STREAM_FAILURE_MESSAGE + yield response_event_stream.emit_failed(message=message, usage=tracker.usage if tracker is not None else None) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py new file mode 100644 index 00000000000..1639c3d575a --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_shared.py @@ -0,0 +1,2238 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Share Responses wire conversion, streaming, and validation between agent and workflow hosts.""" + +from __future__ import annotations + +import asyncio +import base64 +import json +import logging +import uuid +from collections.abc import ( + AsyncGenerator, + AsyncIterator, + Awaitable, + Callable, + Generator, + Mapping, + Sequence, +) +from contextlib import suppress +from dataclasses import asdict, dataclass, is_dataclass +from decimal import Decimal +from typing import Any, Generic, Literal, TypeVar, cast + +from agent_framework import ( + AgentResponse, + AgentResponseUpdate, + AgentSession, + ComputerSafetyCheck, + Content, + FinishReason, + Message, + ResponseStream, + SessionStore, + UsageDetails, + add_usage_details, + get_namespaced_value, +) +from agent_framework import ( + Annotation as FrameworkAnnotation, +) +from agent_framework._mcp import _MCP_TOOL_RESULT_HOST_PAYLOAD_KEY # pyright: ignore[reportPrivateUsage] +from agent_framework._tools import ( + _bind_approval_response_to_pending_request, # pyright: ignore[reportPrivateUsage] + _load_pending_approval_requests, # pyright: ignore[reportPrivateUsage] +) +from agent_framework._types import _get_annotation_text_span_regions # pyright: ignore[reportPrivateUsage] +from agent_framework.observability import OBSERVABILITY_SETTINGS +from azure.ai.agentserver.responses import ( + ResponseContext, +) +from azure.ai.agentserver.responses._id_generator import IdGenerator +from azure.ai.agentserver.responses.aio import ResponseEventStream +from azure.ai.agentserver.responses.models import ( + Annotation as ResponsesAnnotation, +) +from azure.ai.agentserver.responses.models import ( + ComputerAction, + ComputerCallSafetyCheckParam, + ComputerScreenshotImage, + ContainerFileCitationBody, + CreateResponse, + FileCitationBody, + FilePath, + FunctionShellAction, + FunctionShellCallOutputContent, + FunctionShellCallOutputExitOutcome, + FunctionShellCallOutputTimeoutOutcome, + Item, + ItemReasoningItem, + LocalEnvironmentResource, + MessageContent, + OAuthConsentRequestOutputItem, + OutputItem, + OutputItemComputerToolCall, + OutputItemComputerToolCallOutput, + OutputItemMcpApprovalRequest, + OutputItemReasoningItem, + OutputMessageContent, + ResponseIncompleteReason, + ResponseStreamEvent, + ResponseUsage, + ResponseUsageInputTokensDetails, + ResponseUsageOutputTokensDetails, + UrlCitationBody, +) +from azure.ai.agentserver.responses.streaming._builders import ( + OutputItemBuilder, + OutputItemFunctionCallBuilder, + OutputItemMcpCallBuilder, + OutputItemMessageBuilder, + ReasoningSummaryPartBuilder, + RefusalContentBuilder, + TextContentBuilder, +) + +from ._toolbox import _is_allowed_oauth_consent_link # pyright: ignore[reportPrivateUsage] + +logger = logging.getLogger(__name__) + +_MODEL_OUTPUT_KIND_KEY = "model_output_kind" +_MODEL_OUTPUT_REFUSAL = "refusal" +_NATIVE_APPROVAL_ID_KEY = "agent_framework_approval_id" +_NATIVE_APPROVAL_SESSION_KEY = "agent_framework_approval_session_id" +_HOSTED_PROVIDER_STATE_KEY = "_foundry_provider_background" +_HOSTED_SOURCE_CONVERSATION_KEY = "_foundry_source_conversation" +_HOSTED_SERVICE_CHILD_KEY = "_foundry_service_child" +_HOSTED_CONVERSATION_CLAIM_KEY = "_foundry_conversation_claim" +_HOSTED_CONVERSATION_COMMITTED_KEY = "_foundry_conversation_committed" +_HOSTED_PROVIDER_OUTPUT_COUNT_KEY = "_foundry_provider_output_count" +_HOSTED_PROVIDER_USAGE_KEY = "_foundry_provider_usage" +_PENDING_CONTAINER_FILE_CITATIONS_KEY = "_foundry_pending_container_file_citations" +_CONTAINER_FILE_CITATIONS_KEY = "container_file_citations" +_PUBLIC_STREAM_FAILURE_MESSAGE = "An internal server error occurred." + + +class _PublicResponseFailure(Exception): + """Mark a vetted failure message as safe for client-visible serialization.""" + + def __init__(self, error: Exception, public_message: str | None = None) -> None: + super().__init__(public_message or str(error) or type(error).__name__) + self.error = error + self.public_message = public_message or str(error) or type(error).__name__ + + +async def _set_agent_session( # pyright: ignore[reportUnusedFunction] + store: SessionStore, session_id: str, session: AgentSession +) -> None: + """Persist a session while preserving the established public conflict contract.""" + try: + await store.set(session_id, session) + except RuntimeError as exc: + if str(exc) == "Another request advanced this agent session; reload before writing.": + raise _PublicResponseFailure(exc) from None + raise + + +def _log_response_failure( # pyright: ignore[reportUnusedFunction] + phase: str, response_id: str, exc: BaseException +) -> None: + """Log a correlated hosted response failure under the sensitive-data policy.""" + diagnostic_exception = exc.error if isinstance(exc, _PublicResponseFailure) else exc + if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED: + logger.error( + "Hosted Responses failure (phase=%s, response_id=%s, error_type=%s).", + phase, + response_id, + type(diagnostic_exception).__name__, + exc_info=( + type(diagnostic_exception), + diagnostic_exception, + diagnostic_exception.__traceback__, + ), + ) + else: + logger.error( + "Hosted Responses failure (phase=%s, response_id=%s, error_type=%s).", + phase, + response_id, + type(diagnostic_exception).__name__, + ) + + +@dataclass(frozen=True) +class _ContainerFileCitation: + container_id: str + file_id: str + filename: str + + +def _has_container_filename_boundaries(text: str, start_index: int, end_index: int) -> bool: + """Return whether a filename match is not embedded in another filename-like token.""" + before = text[start_index - 1] if start_index > 0 else None + after = text[end_index] if end_index < len(text) else None + return (before is None or not (before.isalnum() or before in "._-")) and ( + after is None or not (after.isalnum() or after in "._-") + ) + + +def _container_file_citations_from_function_result(content: Content) -> list[_ContainerFileCitation]: + """Extract validated container file citations from a core-preserved MCP Host payload.""" + raw_payload = content.additional_properties.get(_MCP_TOOL_RESULT_HOST_PAYLOAD_KEY) + if not isinstance(raw_payload, Mapping): + return [] + payload = cast(Mapping[str, Any], raw_payload) + + metadata_blocks: list[Mapping[str, Any]] = [] + raw_result_meta = payload.get("_meta") + if isinstance(raw_result_meta, Mapping): + metadata_blocks.append(cast(Mapping[str, Any], raw_result_meta)) + + raw_content = payload.get("content") + if isinstance(raw_content, Sequence) and not isinstance(raw_content, (str, bytes, bytearray)): + for raw_item in cast(Sequence[Any], raw_content): + if not isinstance(raw_item, Mapping): + continue + item = cast(Mapping[str, Any], raw_item) + raw_item_meta = item.get("_meta") + if isinstance(raw_item_meta, Mapping): + metadata_blocks.append(cast(Mapping[str, Any], raw_item_meta)) + + citations: list[_ContainerFileCitation] = [] + for metadata in metadata_blocks: + raw_citations = metadata.get(_CONTAINER_FILE_CITATIONS_KEY) + if isinstance(raw_citations, str): + try: + raw_citations = json.loads(raw_citations) + except (json.JSONDecodeError, RecursionError): + continue + + citation_values: list[Any] + if isinstance(raw_citations, Mapping): + citation_values = [cast(Mapping[str, Any], raw_citations)] + elif isinstance(raw_citations, Sequence) and not isinstance(raw_citations, (str, bytes, bytearray)): + citation_values = list(cast(Sequence[Any], raw_citations)) + else: + continue + + fallback_container_id = metadata.get("container_id") + for raw_citation in citation_values: + if not isinstance(raw_citation, Mapping): + continue + citation = cast(Mapping[str, Any], raw_citation) + container_id = citation.get("container_id") or fallback_container_id + file_id = citation.get("file_id") + filename = citation.get("filename") + if not isinstance(container_id, str) or not container_id: + continue + if not isinstance(file_id, str) or not file_id: + continue + if not isinstance(filename, str) or not filename: + continue + citations.append( + _ContainerFileCitation( + container_id=container_id, + file_id=file_id, + filename=filename, + ) + ) + + return citations + + +def _annotations_to_output_text( + annotations: Sequence[FrameworkAnnotation] | None, +) -> list[ResponsesAnnotation]: + """Convert framework citations into Responses output-text annotations. + + Source-coordinate regions describe the cited source, not generated content, and + are not emitted as output-text spans. + """ + if not annotations or not isinstance(annotations, Sequence) or isinstance(annotations, (str, bytes)): + return [] + + output_annotations: list[ResponsesAnnotation] = [] + for annotation in annotations: + if not isinstance(annotation, Mapping): + continue + annotation_mapping = cast("Mapping[str, Any]", annotation) + if annotation_mapping.get("type") != "citation": + continue + + raw_properties = annotation_mapping.get("additional_properties") + properties: Mapping[str, Any] = ( + cast("Mapping[str, Any]", raw_properties) if isinstance(raw_properties, Mapping) else {} + ) + regions = _get_annotation_text_span_regions(annotation_mapping) + file_id = annotation_mapping.get("file_id") + url = annotation_mapping.get("url") + title = annotation_mapping.get("title") + container_id = properties.get("container_id") + index = properties.get("index") + + if isinstance(container_id, str) and container_id and isinstance(file_id, str) and file_id: + for region in regions: + output_annotations.append( + ContainerFileCitationBody( + type="container_file_citation", + container_id=container_id, + file_id=file_id, + filename=url if isinstance(url, str) else "", + start_index=region["start_index"], + end_index=region["end_index"], + ) + ) + elif isinstance(url, str) and url and not file_id: + for region in regions: + output_annotations.append( + UrlCitationBody( + type="url_citation", + url=url, + title=title if isinstance(title, str) else "", + start_index=region["start_index"], + end_index=region["end_index"], + ) + ) + elif ( + isinstance(file_id, str) and file_id and isinstance(url, str) and url and type(index) is int and index >= 0 + ): + output_annotations.append( + FileCitationBody( + type="file_citation", + file_id=file_id, + filename=url, + index=index, + ) + ) + elif isinstance(file_id, str) and file_id and type(index) is int and index >= 0: + output_annotations.append( + FilePath( + type="file_path", + file_id=file_id, + index=index, + ) + ) + + return output_annotations + + +def _is_refusal_text_content(content: Content) -> bool: + return ( + content.type == "text" + and get_namespaced_value( + content.additional_properties, + "core", + _MODEL_OUTPUT_KIND_KEY, + content.additional_properties.get(_MODEL_OUTPUT_KIND_KEY), + ) + == _MODEL_OUTPUT_REFUSAL + ) + + +def _refusal_additional_properties() -> dict[str, Any]: + return { + "core": {_MODEL_OUTPUT_KIND_KEY: _MODEL_OUTPUT_REFUSAL}, + _MODEL_OUTPUT_KIND_KEY: _MODEL_OUTPUT_REFUSAL, + } + + +def _openai_value(content: Content, key: str, *legacy_keys: str, default: Any = None) -> Any: + value = get_namespaced_value(content.additional_properties, "openai", key, None) + if value is not None: + return value + for legacy_key in legacy_keys: + if legacy_key in content.additional_properties: + return content.additional_properties[legacy_key] + return default + + +def _reject_busy_conversation(session: AgentSession) -> None: # pyright: ignore[reportUnusedFunction] + if session.state.get(_HOSTED_CONVERSATION_CLAIM_KEY) is not None: + raise _PublicResponseFailure( + RuntimeError( + "A service-backed conversation already has an in-flight turn. Wait for it to finish; " + "if it failed or was cancelled after dispatch, start a new conversation rather than " + "reusing a possibly changed provider thread." + ) + ) + + +def _request_without_reserved_metadata(request: CreateResponse) -> CreateResponse: + """Copy a create request with any client-supplied reserved metadata bag removed. + + ``ResponseEventStream`` seeds the response from the request, and host recovery later trusts + that bag for incomplete status, provider cursors, usage, and citations. Only a persisted + response is a trusted source. The caller's request mapping is left unchanged. + """ + raw_metadata = request.get("metadata") + if not isinstance(raw_metadata, Mapping): + return request + # Metadata is a TypedDict of known public keys. Cast so a client-supplied reserved key is visible. + metadata = cast(Mapping[str, Any], raw_metadata) + if _RESERVED_RESPONSE_METADATA_KEY not in metadata: + return request + public_metadata: dict[str, Any] = { + key: value for key, value in metadata.items() if key != _RESERVED_RESPONSE_METADATA_KEY + } + seeded: dict[str, Any] = dict(request) + if public_metadata: + seeded["metadata"] = public_metadata + else: + seeded.pop("metadata", None) + return cast(CreateResponse, seeded) + + +def _create_response_event_stream( # pyright: ignore[reportUnusedFunction] + context: ResponseContext, request: CreateResponse +) -> ResponseEventStream: + """Create a response stream from recovery state or the originating request.""" + if context.is_recovery: + persisted_response = context.persisted_response + if persisted_response is not None: + return ResponseEventStream(response=persisted_response, response_id=context.response_id) + return ResponseEventStream( + request=_request_without_reserved_metadata(request), + response_id=context.response_id, + ) + + +def _hosted_internal_metadata(stream: ResponseEventStream) -> dict[str, Any]: + """Keep private state in the durable envelope, outside the public metadata quota. + + The SDK strips its reserved ``internal_metadata`` key at every wire boundary. Older + checkpoints used its quota-limited response metadata bag; copy that trusted state only + when the envelope has no private bag of its own. + """ + metadata = stream.response.get("internal_metadata") + if metadata is None: + metadata = dict(stream.internal_metadata) + stream.response["internal_metadata"] = metadata + if not isinstance(metadata, dict): + raise RuntimeError("The persisted host metadata is invalid.") + return cast(dict[str, Any], metadata) + + +def _read_persisted_incomplete_reason(stream: ResponseEventStream) -> str | None: + """Return a truncating finish reason restored from a trusted response snapshot.""" + persisted = _hosted_internal_metadata(stream).get(_INCOMPLETE_REASON_KEY) + if isinstance(persisted, str): + return persisted + details = stream.response.get("incomplete_details") + if isinstance(details, Mapping): + reason = cast(Mapping[str, Any], details).get("reason") + if isinstance(reason, str): + return reason + return None + + +def _agent_response_updates( # pyright: ignore[reportUnusedFunction] + response: AgentResponse[Any], response_id: str +) -> list[AgentResponseUpdate]: + """Project a completed inner response without exposing its private provider token or ID.""" + updates = [ + AgentResponseUpdate( + contents=list(message.contents), + role=message.role, + author_name=message.author_name, + response_id=response_id, + message_id=message.message_id or uuid.uuid4().hex, + ) + for message in response.messages + ] + if response.usage_details is not None: + updates.append(AgentResponseUpdate(contents=[Content.from_usage(response.usage_details)])) + if response.finish_reason is not None: + if not updates: + updates.append(AgentResponseUpdate()) + updates[-1].finish_reason = FinishReason(response.finish_reason) + return updates + + +_T = TypeVar("_T") + + +async def _await_before_signal( # pyright: ignore[reportUnusedFunction] + operation: Callable[[], Awaitable[_T]], *signals: asyncio.Event +) -> tuple[bool, _T | None]: + """Race a provider await against lifecycle signals without discarding a completed continuation token.""" + if any(signal.is_set() for signal in signals): + return False, None + task = asyncio.ensure_future(operation()) + waiters = [asyncio.ensure_future(signal.wait()) for signal in signals] + try: + finished, _ = await asyncio.wait([task, *waiters], return_when=asyncio.FIRST_COMPLETED) + if task in finished: + return True, await task + return False, None + finally: + if not task.done(): + task.cancel() + with suppress(asyncio.CancelledError): + await task + for waiter in waiters: + waiter.cancel() + await asyncio.gather(*waiters, return_exceptions=True) + + +# Sentinel put on the internal queue by _SignalledIterator's driver task to signal that the +# wrapped iterator is exhausted (distinct from `None`, which is a valid item value). +_STOP_SENTINEL: Any = object() + + +class _SignalledIterator(Generic[_T]): + """Wraps an async iterator, stopping early as soon as any of ``events`` fires. + + Plain ``async for update in agent.run(...): if event.is_set(): break`` only observes ``event`` + once ``run()`` actually yields an item -- if it's suspended on a single slow model or tool call + with no intermediate item, the signal is invisible until that call resolves. This drives the + wrapped iterator from a single persistent background task and races each produced item against + ``events`` via ``asyncio.wait`` instead, so a signal is observed immediately. + + The background task (``_drive``) is required (rather than spawning a fresh task per step) + because some cleanup run by the wrapped iterator (e.g. observability span teardown) resets a + contextvar token set on an earlier call and requires every call against it to share the same + async context. + + If an event and a new item becomes ready at the same time, the event takes priority and the item + is discarded. Cancelling the background task while it's mid-call interrupts a suspended model or + tool call, then the driver closes the underlying stream in ``finally``. + + Callers MUST drive this through ``contextlib.aclosing`` (or an equivalent try/finally calling + ``aclose()``): ``__anext__`` only cancels the driver task on its own signalled/exhausted paths, so + if the consumer of ``async for`` raises instead (e.g. while processing a yielded item), the driver + task -- and the real agent/workflow run it's pumping -- would otherwise be silently abandoned. + """ + + def __init__( + self, + iterator: AsyncIterator[_T], + *events: asyncio.Event, + stamp: Callable[[], Awaitable[Any]] | None = None, + ) -> None: + """Wrap an async iterator, stopping early if any of ``events`` fires. + + Args: + iterator: The async iterator to wrap. + events: One or more asyncio.Event objects to watch for. If any of them is set, iteration stops early. + stamp: Optional coroutine function the driver awaits right after the wrapped iterator produces an + item and before it is advanced again. Its result is exposed as :attr:`stamp` while that item + is the current one, which lets a consumer observe state (e.g. the latest persisted workflow + checkpoint) as it was when the item was produced rather than when it is consumed: the driver + runs one item ahead, so by consumption time the wrapped iterator may already have moved on. + """ + self._iterator = iterator + self._events = events + self._stamp_fn = stamp + self._stamp: Any = None + self._signalled = False + # The queue is used to communicate items from the background driver task to the main iteration loop. + self._queue: asyncio.Queue[Any] = asyncio.Queue(maxsize=1) + # The background task that drives the wrapped iterator. + self._driver: asyncio.Task[None] | None = None + + @property + def signalled(self) -> bool: + """Whether iteration stopped early due to an event being set. + + ``signalled`` is only set when iteration stopped early because of an event -- never on ordinary + exhaustion -- so callers can tell "the agent/workflow finished" apart from "we gave up waiting". + """ + return self._signalled + + @property + def stamp(self) -> Any: + """The ``stamp`` result taken when the current item was produced (``None`` without a ``stamp``).""" + return self._stamp + + def __aiter__(self) -> _SignalledIterator[_T]: + return self + + async def _drive(self) -> None: + """Pull items from the wrapped iterator into ``self._queue`` for the object's lifetime.""" + try: + while True: + try: + item: Any = await self._iterator.__anext__() + stamp = await self._stamp_fn() if self._stamp_fn is not None else None + except StopAsyncIteration: + await self._queue.put(_STOP_SENTINEL) + return + except Exception as exc: + await self._queue.put(exc) + return + await self._queue.put((item, stamp)) + finally: + iterator: AsyncIterator[_T] = self._iterator + if isinstance(iterator, ResponseStream): + await cast(ResponseStream[_T, Any], iterator).close() + else: + close = getattr(iterator, "aclose", None) + if close is not None: + await close() + + async def __anext__(self) -> _T: + if self._driver is None: + self._driver = asyncio.ensure_future(self._drive()) + + # Create the background tasks for monitoring the events and the queue. + waiters = [asyncio.ensure_future(event.wait()) for event in self._events] + get_task = asyncio.ensure_future(self._queue.get()) + try: + # Waits until at least one of the tasks is completed. + await asyncio.wait([get_task, *waiters], return_when=asyncio.FIRST_COMPLETED) + if any(waiter.done() for waiter in waiters): + self._signalled = True + self._driver.cancel() + with suppress(BaseException): + await self._driver + get_task.cancel() + with suppress(BaseException): + await get_task + raise StopAsyncIteration + item = get_task.result() + finally: + for waiter in waiters: + if not waiter.done(): + waiter.cancel() + for waiter in waiters: + with suppress(BaseException): + await waiter + if item is _STOP_SENTINEL: + raise StopAsyncIteration + if isinstance(item, Exception): + raise item + item, self._stamp = item + return cast(_T, item) + + async def aclose(self) -> None: + """Cancel the background driver task, if any, and wait for it to finish. + + Safe to call unconditionally: a no-op if the driver was never started, and cancelling an + already-finished task (normal exhaustion or a prior signalled stop) is also a no-op. + """ + if self._driver is None: + return + self._driver.cancel() + with suppress(BaseException): + await self._driver + + +# Reserved response metadata key pinning the workflow checkpoint that was current at the moment of +# the last successfully persisted response-stream checkpoint. Recovery MUST resume from this specific +# checkpoint if it exists, not simply the latest one in checkpoint_storage: the workflow may have +# saved further checkpoints after it but before a crash, without their output ever being durably +# recorded in response.output. If this key is missing, the workflow will resume from the latest +# checkpoint in storage (if any), or replay the original input if none exists as no output was ever +# durably persisted. +_LATEST_CHECKPOINT_ID_KEY = "_last_checkpoint_id" +# ``internal_metadata`` key carrying a truncating finish reason across resilient checkpoints, so a +# turn cut short before a crash still ends ``incomplete`` after recovery. +_INCOMPLETE_REASON_KEY = "_incomplete_reason" +# SDK-reserved response metadata entry. It round-trips server-owned recovery state (this reason, +# provider cursors, usage, and citations). Clients may send the same key; it is never trusted on a +# fresh request. Foundry counts it toward the 16-entry public metadata limit. +_RESERVED_RESPONSE_METADATA_KEY = "_internal_metadata" + + +# region Active Builder State + + +class _OutputItemTracker: # pyright: ignore[reportUnusedClass] + """Converts a stream of agent ``Content`` into ``ResponseStreamEvent``s for one response. + + For content types that arrive as a series of deltas (text, reasoning, function calls, MCP + calls) it tracks the single currently-open output item builder, merging consecutive same-item + deltas and closing the builder (emitting its `*_done` events) as soon as a different item + starts. All other content types (function results, image generation, shell calls/results, + approval requests, etc.) are emitted in one shot, closing any still-open streaming item first. + """ + + def __init__( + self, + stream: ResponseEventStream, + allowed_oauth_consent_origins: frozenset[str] | None = None, + ) -> None: + self._stream = stream + self._allowed_oauth_consent_origins = allowed_oauth_consent_origins + self._usage_details: UsageDetails | None = None + persisted_usage = _hosted_internal_metadata(stream).get(_HOSTED_PROVIDER_USAGE_KEY) + if persisted_usage is not None: + if not isinstance(persisted_usage, Mapping) or any( + not isinstance(key, str) or (value is not None and type(value) is not int) + for key, value in cast(Mapping[object, object], persisted_usage).items() + ): + raise RuntimeError("The persisted provider usage is invalid.") + self._usage_details = cast(UsageDetails, dict(cast(Mapping[str, int | None], persisted_usage))) + self._active_type: str | None = None + self._active_id: str | None = None + # message_id of the update that opened the active text item, used to detect a new + # logical message (e.g. a fresh workflow yield_output call) even when the content + # type doesn't change, so it isn't silently merged into the still-open item. + self._active_message_id: str | None = None + # Accumulated delta text for the current active builder + self._accumulated: list[str] = [] + # Builder state — only one is active at a time + self._message_item: OutputItemMessageBuilder | None = None + self._text_content: TextContentBuilder | None = None + self._refusal_content: RefusalContentBuilder | None = None + self._reasoning_item: OutputItemBuilder | None = None + self._summary_part: ReasoningSummaryPartBuilder | None = None + self._reasoning_encrypted_content: str | None = None + self._fc_builder: OutputItemFunctionCallBuilder | None = None + self._mcp_builder: OutputItemMcpCallBuilder | None = None + self._outstanding_function_calls: dict[str, str | None] = {} + self._outstanding_computer_calls: set[str] = set() + self._oauth_consent_requests: set[tuple[str, str]] = set() + self._pending_container_file_citations: dict[str, _ContainerFileCitation] = {} + self._text_annotations: list[ResponsesAnnotation] = [] + self._message_text_annotations: dict[int, list[ResponsesAnnotation]] = {} + # Set when an agent update reports the model stopped early (content filter, token + # limit); the response then ends as ``incomplete`` instead of ``completed`` so callers + # can tell a cut-short turn from a successful one. Mirrored onto the response so it + # survives a resilient checkpoint/recovery cycle, which rebuilds this tracker from the + # persisted response. + self._incomplete_reason: ResponseIncompleteReason | None = None + persisted_reason = _read_persisted_incomplete_reason(stream) + if persisted_reason is not None: + with suppress(ValueError): + self._incomplete_reason = ResponseIncompleteReason(persisted_reason) + persisted_citations = _hosted_internal_metadata(stream).get(_PENDING_CONTAINER_FILE_CITATIONS_KEY) + if persisted_citations is not None: + if not isinstance(persisted_citations, Sequence) or isinstance( + persisted_citations, (str, bytes, bytearray) + ): + raise RuntimeError("The persisted container file citations are invalid.") + for raw_citation in cast(Sequence[Any], persisted_citations): + if not isinstance(raw_citation, Mapping): + raise RuntimeError("The persisted container file citations are invalid.") + citation = cast(Mapping[str, Any], raw_citation) + container_id = citation.get("container_id") + file_id = citation.get("file_id") + filename = citation.get("filename") + if ( + not isinstance(container_id, str) + or not container_id + or not isinstance(file_id, str) + or not file_id + or not isinstance(filename, str) + or not filename + ): + raise RuntimeError("The persisted container file citations are invalid.") + self._pending_container_file_citations[filename] = _ContainerFileCitation( + container_id=container_id, + file_id=file_id, + filename=filename, + ) + for item in stream.response.get("output", []): + if not isinstance(item, Mapping): + continue + persisted_item = cast(Mapping[str, Any], item) + if persisted_item.get("type") == "computer_call": + call_id = persisted_item.get("call_id") + if isinstance(call_id, str): + self._outstanding_computer_calls.add(call_id) + elif persisted_item.get("type") == "computer_call_output": + if isinstance(call_id := persisted_item.get("call_id"), str): + self._outstanding_computer_calls.discard(call_id) + if persisted_item.get("type") != "oauth_consent_request": + continue + consent_link = persisted_item.get("consent_link") + server_label = persisted_item.get("server_label") + if isinstance(consent_link, str) and isinstance(server_label, str): + self._oauth_consent_requests.add((consent_link, server_label)) + + @property + def usage_details(self) -> UsageDetails | None: + """Return usage retained with the provider output checkpoint.""" + return self._usage_details + + @property + def usage(self) -> ResponseUsage | None: + """Return accumulated usage in the Responses API schema.""" + if self._usage_details is None: + return None + + input_tokens = int(self._usage_details.get("input_token_count") or 0) + output_tokens = int(self._usage_details.get("output_token_count") or 0) + total_tokens = self._usage_details.get("total_token_count") + return ResponseUsage( + input_tokens=input_tokens, + input_tokens_details=ResponseUsageInputTokensDetails( + cached_tokens=int(self._usage_details.get("cache_read_input_token_count") or 0), + cache_write_tokens=int(self._usage_details.get("cache_creation_input_token_count") or 0), + ), + output_tokens=output_tokens, + output_tokens_details=ResponseUsageOutputTokensDetails( + reasoning_tokens=int(self._usage_details.get("reasoning_output_token_count") or 0) + ), + total_tokens=int(total_tokens) if total_tokens is not None else input_tokens + output_tokens, + ) + + @property + def oauth_consent_requested(self) -> bool: + """Return whether this response emitted an OAuth consent request.""" + return bool(self._oauth_consent_requests) + + @property + def incomplete_reason(self) -> ResponseIncompleteReason | None: + """Return why the turn was cut short, if any update reported a truncating finish reason.""" + return self._incomplete_reason + + def record_finish_reason(self, finish_reason: str | None) -> None: + """Note the finish reason of an agent update. + + Only finish reasons that mean the model stopped early are retained, mapped onto the + Responses ``incomplete_details.reason`` vocabulary. A content filter is kept in + preference to a token limit if both are seen during a multi-step turn, since it is the + more actionable signal for the caller. The reason is recorded even when the caller's + metadata already fills the public map, and that map is left intact. + """ + if finish_reason == "content_filter": + self._incomplete_reason = ResponseIncompleteReason.CONTENT_FILTER + elif finish_reason == "length" and self._incomplete_reason is None: + self._incomplete_reason = ResponseIncompleteReason.MAX_OUTPUT_TOKENS + else: + return + _hosted_internal_metadata(self._stream)[_INCOMPLETE_REASON_KEY] = self._incomplete_reason.value + + def _consume_text_update(self, text: str, merge_mode: Literal["append", "replace"] | None) -> str: + """Update the authoritative text buffer and return the wire delta still needed.""" + if merge_mode != "replace": + self._accumulated.append(text) + return text + + previous = "".join(self._accumulated) + if text.startswith(previous): + self._accumulated[:] = [text] + return text[len(previous) :] + if not previous: + self._accumulated[:] = [text] + return text + raise _PublicResponseFailure( + RuntimeError("Cannot project a divergent replacement text snapshot onto an append-only Responses stream.") + ) + + async def handle_update( + self, + update: AgentResponseUpdate, + *, + complete_message: bool = False, + approval_session_id: str | None = None, + ) -> AsyncGenerator[ResponseStreamEvent]: + """Process one agent update: note its finish reason, then handle each of its contents. + + This is the single entry point for both the plain-agent and the workflow loops, so the + finish reason cannot be forgotten on one of them. Set ``complete_message`` when the + update contains a completed message's independent content parts rather than streaming + deltas; text and refusal parts then retain their original boundaries and citation offsets. + """ + self.record_finish_reason(update.finish_reason) + for content in update.contents: + async for event in self.handle( + content, + message_id=update.message_id, + start_new_content_part=complete_message, + approval_session_id=approval_session_id, + ): + yield event + + async def handle( + self, + content: Content, + message_id: str | None = None, + *, + start_new_content_part: bool = False, + approval_session_id: str | None = None, + ) -> AsyncGenerator[ResponseStreamEvent]: + """Process a content item, yielding its events. + + Args: + content: The content item to process. + message_id: The ``message_id`` of the update ``content`` came from, if any. A + change in ``message_id`` across otherwise same-typed text content marks a new + logical message and forces the previous output item closed, rather than being + merged into it. + start_new_content_part: Whether text and refusal content start a new independent + content part instead of continuing the current streaming occurrence. + approval_session_id: Native session identity used only to scope stored wire correlation. + """ + if _is_refusal_text_content(content) and content.text is not None: + for event in self._ensure_message_content("refusal", message_id, start_new_part=start_new_content_part): + yield event + self._active_message_id = message_id + delta = self._consume_text_update(content.text, content.merge_mode) + if delta and self._refusal_content is not None: + yield self._refusal_content.emit_delta(delta) + + elif content.type == "text" and content.text is not None: + for event in self._ensure_message_content("text", message_id, start_new_part=start_new_content_part): + yield event + self._active_message_id = message_id + delta = self._consume_text_update(content.text, content.merge_mode) + for annotation in _annotations_to_output_text(content.annotations): + if annotation not in self._text_annotations: + self._text_annotations.append(annotation) + if delta and self._text_content is not None: + yield self._text_content.emit_delta(delta) + + elif content.type == "text_reasoning": + if self._active_type != "text_reasoning" or (content.id is not None and content.id != self._active_id): + for event in self._close(): + yield event + for event in self._open_reasoning(content): + yield event + if encrypted_content := _reasoning_encrypted_content(content): + self._reasoning_encrypted_content = encrypted_content + if content.text: + delta = self._consume_text_update(content.text, content.merge_mode) + if delta and self._summary_part is not None: + yield self._summary_part.emit_text_delta(delta) + + elif content.type == "function_call" and content.call_id is not None: + # Declaration-only calls replay request metadata after the streamed call. Scope suppression to the + # outstanding occurrence because a call_id may be reused after its terminal result. + if ( + content.user_input_request + and content.arguments is None + and content.call_id in self._outstanding_function_calls + and self._outstanding_function_calls[content.call_id] == content.name + ): + return + if self._active_type != "function_call" or self._active_id != content.call_id: + for event in self._close(): + yield event + for event in self._open_function_call(content): + yield event + args_str = _json_safe_to_str(content.arguments) + delta = self._consume_text_update(args_str, content.merge_mode) + if delta and self._fc_builder is not None: + yield self._fc_builder.emit_arguments_delta(delta) + + elif content.type == "function_result": + for event in self._close(): + yield event + citations = _container_file_citations_from_function_result(content) + if citations: + for citation in citations: + self._pending_container_file_citations[citation.filename] = citation + self._persist_pending_container_file_citations() + async for event in self._stream.output_item_function_call_output( + content.call_id, # type: ignore[arg-type] + _json_safe_to_str(content.result), + ): + yield event + if content.call_id is not None: + self._outstanding_function_calls.pop(content.call_id, None) + + elif content.type == "mcp_server_tool_call" and content.tool_name: + key = content.call_id or f"{content.server_name or 'default'}::{content.tool_name}" + if self._active_type != "mcp_server_tool_call" or self._active_id != key: + for event in self._close(): + yield event + for event in self._open_mcp_call(content): + yield event + args_str = _json_safe_to_str(content.arguments) + delta = self._consume_text_update(args_str, content.merge_mode) + if delta and self._mcp_builder is not None: + yield self._mcp_builder.emit_arguments_delta(delta) + + elif ( + content.type == "mcp_server_tool_result" + and self._active_type == "mcp_server_tool_call" + and self._mcp_builder is not None + and content.call_id is not None + and content.call_id == self._mcp_builder.item_id + ): + accumulated = "".join(self._accumulated) + yield self._mcp_builder.emit_arguments_done(accumulated) + yield self._mcp_builder.emit_completed() + yield self._mcp_builder.emit_done(output=_stringify_mcp_output(content.output)) + self._mcp_builder = None + self._active_type = None + self._active_id = None + self._accumulated.clear() + return + + elif content.type == "image_generation_tool_result" and content.outputs is not None: + for event in self._close(): + yield event + result = _image_generation_result(content.outputs) + if result is None: + logger.warning("Image generation result has no base64 image; emitting it with an empty result.") + async for event in self._stream.output_item_image_gen_call(result or ""): + yield event + + elif content.type == "mcp_server_tool_call": + # Reached only when `content.tool_name` is falsy (the streaming branch above didn't match). + for event in self._close(): + yield event + mcp_call = self._stream.add_output_item_mcp_call( + server_label=content.server_name or "default", + name=content.tool_name or "", + item_id=content.call_id, + ) + yield mcp_call.emit_added() + async for event in mcp_call.arguments(_json_safe_to_str(content.arguments)): + yield event + yield mcp_call.emit_completed() + yield mcp_call.emit_done() + + elif content.type == "mcp_server_tool_result": + # Reached when there's no correlated in-progress mcp_server_tool_call to close against. + for event in self._close(): + yield event + output = _stringify_mcp_output(content.output) + async for event in self._stream.output_item_custom_tool_call_output(content.call_id or "", output): + yield event + + elif content.type == "shell_tool_call": + for event in self._close(): + yield event + action = FunctionShellAction( + commands=content.commands or [], + timeout_ms=content.timeout_ms, + max_output_length=content.max_output_length, + ) + async for event in self._stream.output_item_function_shell_call( + content.call_id or "", + action, + LocalEnvironmentResource(type="local"), + status=content.status or "completed", + ): + yield event + + elif content.type == "shell_tool_result": + for event in self._close(): + yield event + output_items: list[FunctionShellCallOutputContent] = [] + if content.outputs: + for out in content.outputs: + exit_code = getattr(out, "exit_code", None) + outcome = ( + FunctionShellCallOutputTimeoutOutcome(type="timeout") + if getattr(out, "timed_out", False) + else FunctionShellCallOutputExitOutcome( + type="exit", + exit_code=exit_code if exit_code is not None else 0, + ) + ) + output_items.append( + FunctionShellCallOutputContent( + stdout=getattr(out, "stdout", "") or "", + stderr=getattr(out, "stderr", "") or "", + outcome=outcome, + ) + ) + async for event in self._stream.output_item_function_shell_call_output( + content.call_id or "", + output_items, + status=content.status or "completed", + max_output_length=content.max_output_length, + ): + yield event + + elif content.type == "computer_tool_call": + if not content.id or not content.call_id or not content.actions: + raise ValueError("A computer call requires an item id, call_id, and actions.") + if content.call_id in self._outstanding_computer_calls: + return + for event in self._close(): + yield event + if IdGenerator.is_valid(content.id)[0]: + builder = self._stream.add_output_item(content.id) + else: + logger.warning("Remapping computer call item id %r to an AgentServer id.", content.id) + builder = self._stream.add_output_item_computer_call() + status = content.status if content.status is not None else "completed" + if status not in ("in_progress", "completed", "incomplete"): + raise ValueError(f"Unsupported computer call status: {status!r}") + item = OutputItemComputerToolCall( + type="computer_call", + id=builder.item_id, + call_id=content.call_id, + actions=cast(list[ComputerAction], content.actions), + pending_safety_checks=cast(list[ComputerCallSafetyCheckParam], content.pending_safety_checks or []), + status=status, + ) + if _openai_value(content, "computer_action_format", "computer_action_format") == "single": + if len(content.actions) != 1: + raise ValueError("A preview computer call requires exactly one action.") + item.pop("actions") + item["action"] = cast(ComputerAction, content.actions[0]) + yield builder.emit_added(item) + yield builder.emit_done(item) + self._outstanding_computer_calls.add(content.call_id) + + elif content.type == "computer_tool_result": + if not content.call_id or content.screenshot is None: + raise _PublicResponseFailure(ValueError("A computer result requires a call_id and screenshot.")) + for event in self._close(): + yield event + if content.id and IdGenerator.is_valid(content.id)[0]: + builder = self._stream.add_output_item(content.id) + else: + if content.id: + logger.warning("Remapping computer output item id %r to an AgentServer id.", content.id) + builder = self._stream.add_output_item_computer_call_output() + item = OutputItemComputerToolCallOutput( + type="computer_call_output", + id=builder.item_id, + call_id=content.call_id, + output=cast(ComputerScreenshotImage, _computer_screenshot_to_output(content.screenshot)), + ) + if content.status is not None: + if content.status not in ("in_progress", "completed", "incomplete"): + raise ValueError(f"Unsupported computer output status: {content.status!r}") + item["status"] = content.status + if content.acknowledged_safety_checks is not None: + item["acknowledged_safety_checks"] = cast( + list[ComputerCallSafetyCheckParam], content.acknowledged_safety_checks + ) + yield builder.emit_added(item) + yield builder.emit_done(item) + self._outstanding_computer_calls.discard(content.call_id) + + elif content.type == "function_approval_request": + for event in self._close(): + yield event + function_call = content.function_call + if not content.id or function_call is None or not function_call.name: + raise ValueError("An approval request requires a native request ID and a named function call.") + server_label = function_call.server_name or "agent_framework" + wire_id = IdGenerator.new_mcp_approval_request_item_id(str(self._stream.response["id"])) + item = OutputItemMcpApprovalRequest( + type="mcp_approval_request", + id=wire_id, + server_label=server_label, + name=function_call.name, + arguments=_json_safe_to_str(function_call.arguments), + ) + builder = self._stream.add_output_item(wire_id) + builder.internal_metadata[_NATIVE_APPROVAL_ID_KEY] = content.id + if approval_session_id is not None: + builder.internal_metadata[_NATIVE_APPROVAL_SESSION_KEY] = approval_session_id + yield builder.emit_added(item) + yield builder.emit_done(item) + + elif content.type == "oauth_consent_request": + for event in self._close(): + yield event + + consent_link = content.consent_link + if not _is_allowed_oauth_consent_link(consent_link, self._allowed_oauth_consent_origins): + raise ValueError("OAuth consent request content must include an allowed safe HTTPS consent link.") + + server_label = get_namespaced_value( + content.additional_properties, + "openai", + "server_label", + content.additional_properties.get("server_label"), + ) + if not isinstance(server_label, str) or not server_label: + server_label = getattr(content.raw_representation, "server_label", None) + if not isinstance(server_label, str) or not server_label: + server_label = "agent_framework" + + consent_key = (consent_link, server_label) + if consent_key in self._oauth_consent_requests: + return + self._oauth_consent_requests.add(consent_key) + + oauth_item = OAuthConsentRequestOutputItem( + id=IdGenerator.new_id("oacr"), + response_id=str(self._stream.response["id"]), + type="oauth_consent_request", + consent_link=consent_link, + server_label=server_label, + ) + builder = self._stream.add_output_item(oauth_item["id"]) + yield builder.emit_added(oauth_item) + yield builder.emit_done(oauth_item) + + elif content.type == "usage": + self._usage_details = add_usage_details( + None if content.merge_mode == "replace" else self._usage_details, content.usage_details + ) + + else: + for event in self._close(): + yield event + # Defensive: covers content types not recognized above (e.g. "text"/"text_reasoning"/ + # "function_call" with missing required fields), logged instead of raised so the + # response stream isn't broken by one unsupported content item. + logger.warning(f"Content type '{content.type}' is not supported yet. This is usually safe to ignore.") + + def close(self) -> Generator[ResponseStreamEvent]: + """Flush any remaining active builder without discarding recoverable state.""" + yield from self._close() + + def discard_pending_container_file_citations(self) -> None: + """Discard unmatched citations immediately before a terminal response event.""" + self._pending_container_file_citations.clear() + self._persist_pending_container_file_citations() + + # -- Private open/close helpers -- + + def _ensure_message_content( + self, + content_type: Literal["text", "refusal"], + message_id: str | None, + *, + start_new_part: bool = False, + ) -> Generator[ResponseStreamEvent]: + message_changed = ( + message_id is not None and self._active_message_id is not None and message_id != self._active_message_id + ) + if self._active_type == content_type and not message_changed and not start_new_part: + return + if self._message_item is not None and self._active_type in {"text", "refusal"} and not message_changed: + yield from self._close_message_content() + yield from self._open_message_content(content_type) + return + yield from self._close() + yield from self._open_message(content_type) + + def _open_message(self, content_type: Literal["text", "refusal"]) -> Generator[ResponseStreamEvent]: + self._message_text_annotations.clear() + self._message_item = self._stream.add_output_item_message() + yield self._message_item.emit_added() + yield from self._open_message_content(content_type) + + def _open_message_content( + self, + content_type: Literal["text", "refusal"], + ) -> Generator[ResponseStreamEvent]: + if self._message_item is None: + raise RuntimeError("Cannot open message content without an active message") + self._active_type = content_type + self._active_id = None + if content_type == "refusal": + self._refusal_content = self._message_item.add_refusal_content() + yield self._refusal_content.emit_added() + else: + self._text_content = self._message_item.add_text_content() + self._text_annotations.clear() + yield self._text_content.emit_added() + + def _open_reasoning(self, content: Content) -> Generator[ResponseStreamEvent]: + item_id = content.id + if not item_id or not IdGenerator.is_valid(item_id)[0]: + item_id = IdGenerator.new_id("rs") + self._reasoning_item = self._stream.add_output_item(item_id) + self._summary_part = ReasoningSummaryPartBuilder( + self._stream, + self._reasoning_item.output_index, + 0, + item_id, + ) + self._reasoning_encrypted_content = _reasoning_encrypted_content(content) + self._active_type = "text_reasoning" + self._active_id = item_id + yield self._reasoning_item.emit_added( + _reasoning_output_item( + item_id=item_id, + summary_texts=[], + encrypted_content=None, + status="in_progress", + ) + ) + yield self._summary_part.emit_added() + + def _open_function_call(self, content: Content) -> Generator[ResponseStreamEvent]: + self._fc_builder = self._stream.add_output_item_function_call( + name=content.name or "", + call_id=content.call_id or "", + ) + self._active_type = "function_call" + self._active_id = content.call_id + self._outstanding_function_calls[content.call_id or ""] = content.name + yield self._fc_builder.emit_added() + + def _open_mcp_call(self, content: Content) -> Generator[ResponseStreamEvent]: + self._mcp_builder = self._stream.add_output_item_mcp_call( + server_label=content.server_name or "default", + name=content.tool_name or "", + item_id=content.call_id, + ) + self._active_type = "mcp_server_tool_call" + self._active_id = content.call_id or f"{content.server_name or 'default'}::{content.tool_name}" + yield self._mcp_builder.emit_added() + + def _close(self) -> Generator[ResponseStreamEvent]: + if self._active_type in {"text", "refusal"}: + yield from self._close_message_content() + if self._message_item is not None: + message_done = self._message_item.emit_done() + if self._message_text_annotations: + message_done_dict = cast(dict[str, Any], message_done) + item = cast(dict[str, Any], message_done_dict["item"]) + content_parts = cast(list[dict[str, Any]], item["content"]) + for content_index, annotations in self._message_text_annotations.items(): + if content_index >= len(content_parts): + raise RuntimeError("Container file citation content index is out of range.") + content_parts[content_index]["annotations"] = annotations + + response_output = self._stream.response.get("output") + output_index = self._message_item.output_index + if not isinstance(response_output, list): + raise RuntimeError("Container file citation output index is out of range.") + response_output_items = cast(list[Any], response_output) + if output_index >= len(response_output_items): + raise RuntimeError("Container file citation output index is out of range.") + response_output_items[output_index] = item + yield message_done + self._message_item = None + self._message_text_annotations.clear() + + elif self._active_type == "text_reasoning" and self._summary_part and self._reasoning_item: + accumulated = "".join(self._accumulated) + yield self._summary_part.emit_text_done(accumulated) + yield self._summary_part.emit_done() + yield self._reasoning_item.emit_done( + _reasoning_output_item( + item_id=self._reasoning_item.item_id, + summary_texts=[accumulated], + encrypted_content=self._reasoning_encrypted_content, + status="completed", + ) + ) + self._summary_part = None + self._reasoning_item = None + self._reasoning_encrypted_content = None + + elif self._active_type == "function_call" and self._fc_builder: + accumulated = "".join(self._accumulated) + yield self._fc_builder.emit_arguments_done(accumulated) + yield self._fc_builder.emit_done() + self._fc_builder = None + + elif self._active_type == "mcp_server_tool_call" and self._mcp_builder: + accumulated = "".join(self._accumulated) + yield self._mcp_builder.emit_arguments_done(accumulated) + yield self._mcp_builder.emit_completed() + yield self._mcp_builder.emit_done() + self._mcp_builder = None + + self._active_type = None + self._active_id = None + self._active_message_id = None + self._accumulated.clear() + + def _close_message_content(self) -> Generator[ResponseStreamEvent]: + accumulated = "".join(self._accumulated) + if self._active_type == "text" and self._text_content is not None: + annotations = list(self._text_annotations) + for annotation in self._container_file_annotations_for_text(accumulated): + if annotation not in annotations: + annotations.append(annotation) + yield self._text_content.emit_text_done(accumulated) + for annotation in annotations: + yield self._text_content.emit_annotation_added(annotation) + content_done = self._text_content.emit_done() + if annotations: + content_done_dict = cast(dict[str, Any], content_done) + part = cast(dict[str, Any], content_done_dict["part"]) + part["annotations"] = annotations + self._message_text_annotations[self._text_content.content_index] = annotations + yield content_done + self._text_content = None + self._text_annotations.clear() + elif self._active_type == "refusal" and self._refusal_content is not None: + yield self._refusal_content.emit_refusal_done(accumulated) + yield self._refusal_content.emit_done() + self._refusal_content = None + self._active_type = None + self._active_id = None + self._accumulated.clear() + + def _container_file_annotations_for_text(self, text: str) -> list[ContainerFileCitationBody]: + candidates: list[tuple[int, int, str, _ContainerFileCitation]] = [] + for filename, citation in self._pending_container_file_citations.items(): + search_index = 0 + while True: + start_index = text.find(filename, search_index) + if start_index < 0: + break + end_index = start_index + len(filename) + if _has_container_filename_boundaries(text, start_index, end_index): + candidates.append((start_index, end_index, filename, citation)) + search_index = start_index + 1 + + candidates.sort(key=lambda match: (-(match[1] - match[0]), match[0], match[2])) + selected: list[tuple[int, int, str, _ContainerFileCitation]] = [] + selected_filenames: set[str] = set() + for candidate in candidates: + start_index, end_index, filename, _ = candidate + if filename in selected_filenames: + continue + if any( + start_index < selected_end and selected_start < end_index + for selected_start, selected_end, _, _ in selected + ): + continue + selected.append(candidate) + selected_filenames.add(filename) + + annotations: list[ContainerFileCitationBody] = [] + for start_index, end_index, filename, citation in sorted(selected, key=lambda match: match[0]): + annotations.append( + ContainerFileCitationBody( + type="container_file_citation", + container_id=citation.container_id, + file_id=citation.file_id, + filename=citation.filename, + start_index=start_index, + end_index=end_index, + ) + ) + del self._pending_container_file_citations[filename] + if annotations: + self._persist_pending_container_file_citations() + return annotations + + def _persist_pending_container_file_citations(self) -> None: + if not self._pending_container_file_citations: + _hosted_internal_metadata(self._stream).pop(_PENDING_CONTAINER_FILE_CITATIONS_KEY, None) + return + _hosted_internal_metadata(self._stream)[_PENDING_CONTAINER_FILE_CITATIONS_KEY] = [ + { + "container_id": citation.container_id, + "file_id": citation.file_id, + "filename": citation.filename, + } + for citation in self._pending_container_file_citations.values() + ] + + +# endregion + + +# region Input Message Conversion + + +async def _items_to_messages( # pyright: ignore[reportUnusedFunction] + input_items: Sequence[Item], + *, + session: AgentSession | None = None, + approval_ids: Mapping[str, str] | None = None, + include_hosted_approval_requests: bool = False, +) -> list[Message]: + """Convert input items and restore native context for current approval decisions. + + Args: + input_items: The input items to convert. + + Keyword Args: + session: Native session holding the agent's authoritative pending requests. + approval_ids: Transport-only mapping from stored wire IDs to native request IDs. + include_hosted_approval_requests: Replay hosted requests when hosting owns the stateless + model transcript. Agent-managed or service-side history supplies its own context. + + Returns: + Native request context followed by the supported input messages. + """ + messages: list[Message] = [] + if any(item["type"] == "mcp_approval_response" for item in input_items): + requests = [ + request + for request in _load_pending_approval_requests(session).values() + if request.function_call is not None + and (include_hosted_approval_requests or not request.function_call.hosted) + ] + if requests: + # Capped wire history may omit calls. Local snapshots let Core restore call/result + # pairs; hosted snapshots belong only in the host-managed stateless transcript. + messages.append(Message(role="assistant", contents=requests)) + for item in input_items: + messages.append(await _item_to_message(item, session=session, approval_ids=approval_ids)) + return messages + + +def _reasoning_item_to_contents(reasoning: ItemReasoningItem | OutputItemReasoningItem) -> list[Content]: + """Convert a hosted reasoning item without losing its stateless replay metadata.""" + encrypted_content = reasoning.get("encrypted_content") + if summary_parts := reasoning.get("summary"): + return [ + Content.from_text_reasoning( + id=reasoning["id"], + text=summary["text"], + protected_data=encrypted_content if index == 0 else None, + ) + for index, summary in enumerate(summary_parts) + ] + return [Content.from_text_reasoning(id=reasoning["id"], protected_data=encrypted_content)] + + +def _computer_safety_checks(checks: Sequence[Mapping[str, Any]] | None) -> list[ComputerSafetyCheck] | None: + if checks is None: + return None + parsed: list[ComputerSafetyCheck] = [] + for check in checks: + check_id = check.get("id") + if not isinstance(check_id, str) or not check_id: + raise ValueError("Computer safety checks require an id.") + entry = ComputerSafetyCheck(id=check_id) + for key in ("code", "message"): + value = check.get(key) + if value is not None: + if not isinstance(value, str): + raise ValueError(f"Computer safety check {key} must be a string.") + entry[key] = value + parsed.append(entry) + return parsed + + +def _computer_screenshot_from_output(output: Mapping[str, Any]) -> Content: + if output.get("type") != "computer_screenshot": + raise ValueError("Computer call output must contain a computer screenshot.") + image_url = output.get("image_url") + file_id = output.get("file_id") + openai_properties: dict[str, Any] = {} + legacy_properties: dict[str, Any] = {} + if detail := output.get("detail"): + openai_properties["image_detail"] = detail + legacy_properties["detail"] = detail + additional_properties = { + "openai": openai_properties, + **legacy_properties, + } + if isinstance(image_url, str) and image_url: + return Content.from_uri( + image_url, + file_id=file_id if isinstance(file_id, str) else None, + additional_properties={ + **additional_properties, + **({"file_id": file_id} if isinstance(file_id, str) else {}), + }, + ) + if isinstance(file_id, str) and file_id: + return Content.from_hosted_file(file_id, additional_properties=additional_properties) + raise ValueError("Computer screenshot is missing its image URL or file ID.") + + +def _computer_screenshot_to_output(screenshot: Content) -> dict[str, Any]: + output: dict[str, Any] = {"type": "computer_screenshot"} + if screenshot.type in ("data", "uri") and screenshot.uri: + output["image_url"] = screenshot.uri + if file_id := screenshot.file_id or screenshot.additional_properties.get("file_id"): + output["file_id"] = file_id + elif screenshot.type == "hosted_file" and screenshot.file_id: + output["file_id"] = screenshot.file_id + else: + raise ValueError("A computer screenshot requires image data, a URI, or a hosted file.") + if detail := _openai_value(screenshot, "image_detail", "detail"): + output["detail"] = detail + return output + + +def _shell_command_output_to_content(output: Mapping[str, Any]) -> Content: + outcome = output["outcome"] + outcome_type = outcome.get("type") + return Content.from_shell_command_output( + stdout=output.get("stdout") or "", + stderr=output.get("stderr") or "", + exit_code=outcome.get("exit_code") if outcome_type == "exit" else None, + timed_out=True if outcome_type == "timeout" else False if outcome_type == "exit" else None, + ) + + +async def _item_to_message( # ruff: ignore[unused-async] -- Keep the shared async conversion contract. + item: Item, + *, + session: AgentSession | None = None, + approval_ids: Mapping[str, str] | None = None, + _item_type_name: Literal["Item", "OutputItem"] = "Item", +) -> Message: + """Converts an Item to a Message. + + Args: + item: The Item to convert. + session: Native session holding the agent's authoritative pending requests. + approval_ids: Transport-only mapping from stored wire IDs to native request IDs. + _item_type_name: The item type name to include in unsupported-type errors. + + Returns: + The converted Message. + + Raises: + ValueError: If the Item type is not supported. + """ + if item["type"] == "message": + if isinstance(item["content"], str): + return Message(role=item["role"], contents=[Content.from_text(item["content"])]) + return Message(role=item["role"], contents=[_convert_message_content(part) for part in item["content"]]) + + if item["type"] == "output_message": + return Message(role=item["role"], contents=[_convert_output_message_content(part) for part in item["content"]]) + + if item["type"] == "function_call": + item_data = cast("Mapping[str, Any]", item) + item_id = item_data.get("id") + status = item_data.get("status") + additional_properties: dict[str, Any] = {} + if isinstance(item_id, str): + additional_properties.update({"openai": {"item_id": item_id}, "fc_id": item_id}) + if status is not None: + additional_properties["status"] = status + return Message( + role="assistant", + contents=[ + Content.from_function_call( + item["call_id"], + item["name"], + arguments=item["arguments"], + status=status if isinstance(status, str) else None, + additional_properties=additional_properties, + ) + ], + ) + + if item["type"] == "function_call_output": + call_id = item.get("call_id") + if call_id is None: + raise ValueError("Function call output item is missing a call_id.") + item_id = item.get("id") + result_additional_properties = ( + { + "openai": {"item_id": item_id}, + "item_id": item_id, + } + if isinstance(item_id, str) and item_id + else None + ) + return Message( + role="tool", + contents=[ + Content.from_function_result( + call_id, + result=_json_safe_to_str(item["output"]), + status=item.get("status"), + additional_properties=result_additional_properties, + ) + ], + ) + + if item["type"] == "reasoning": + return Message(role="assistant", contents=_reasoning_item_to_contents(item)) + + if item["type"] == "mcp_call": + contents = [ + Content.from_mcp_server_tool_call( + item["id"], + item["name"], + server_name=item["server_label"], + arguments=item["arguments"], + ) + ] + if (output := item.get("output")) is not None: + contents.append(Content.from_mcp_server_tool_result(call_id=item["id"], output=output)) + return Message( + role="assistant", + contents=contents, + ) + + if item["type"] == "mcp_approval_request": + pending = _load_pending_approval_requests(session) + request_id = approval_ids.get(item["id"]) if approval_ids is not None else item["id"] + function_approval_request_content = pending.get(request_id) if request_id is not None else None + if function_approval_request_content is None: + raise _PublicResponseFailure(ValueError(f"No native pending approval matches {item['id']!r}.")) + return Message( + role="assistant", + contents=[function_approval_request_content], + ) + + if item["type"] == "mcp_approval_response": + request_id = ( + approval_ids.get(item["approval_request_id"]) if approval_ids is not None else item["approval_request_id"] + ) + response = Content( + "function_approval_response", + id=request_id, + approved=item["approve"] is True, + ) + bound = ( + _bind_approval_response_to_pending_request(response, session, consume=False) + if session is not None and request_id is not None + else None + ) + if bound is None: + raise _PublicResponseFailure( + ValueError(f"No native pending approval matches {item['approval_request_id']!r}.") + ) + response.function_call = bound.function_call + return Message( + role="user", + contents=[response], + ) + + if item["type"] == "code_interpreter_call": + return Message( + role="assistant", + contents=[Content.from_code_interpreter_tool_call(call_id=item["id"])], + ) + + if item["type"] == "image_generation_call": + return Message( + role="assistant", + contents=[Content.from_image_generation_tool_call(image_id=item["id"])], + ) + + if item["type"] == "shell_call": + return Message( + role="assistant", + contents=[ + Content.from_shell_tool_call( + call_id=item["call_id"], + commands=item["action"]["commands"], + timeout_ms=item["action"].get("timeout_ms"), + max_output_length=item["action"].get("max_output_length"), + status=str(item.get("status")), + ) + ], + ) + + if item["type"] == "shell_call_output": + outputs = [_shell_command_output_to_content(out) for out in (item["output"] or [])] + return Message( + role="tool", + contents=[ + Content.from_shell_tool_result( + call_id=item["call_id"], + outputs=outputs, + max_output_length=item.get("max_output_length"), + ) + ], + ) + + if item["type"] == "local_shell_call": + commands = item["action"].get("command") or [] + return Message( + role="assistant", + contents=[ + Content.from_shell_tool_call( + call_id=item["call_id"], + commands=commands, + timeout_ms=item["action"].get("timeout_ms"), + status=str(item["status"]), + ) + ], + ) + + if item["type"] == "local_shell_call_output": + return Message( + role="tool", + contents=[ + Content.from_shell_tool_result( + call_id=item["id"], + outputs=[Content.from_shell_command_output(stdout=item["output"])], + ) + ], + ) + + if item["type"] == "file_search_call": + return Message( + role="assistant", + contents=[ + Content.from_function_call( + item["id"], + "file_search", + arguments=_json_safe_to_str({"queries": item["queries"]}), + informational_only=True, + hosted=True, + ) + ], + ) + + if item["type"] == "web_search_call": + return Message( + role="assistant", + contents=[Content.from_function_call(item["id"], "web_search", informational_only=True, hosted=True)], + ) + + if item["type"] == "computer_call": + plural_actions = item.get("actions") + singular_action = item.get("action") + actions = plural_actions or ([singular_action] if singular_action is not None else []) + if not actions: + raise ValueError("Computer call is missing its ordered actions.") + props = ( + { + "openai": {"computer_action_format": "single"}, + "computer_action_format": "single", + } + if not plural_actions and singular_action is not None + else None + ) + return Message( + role="assistant", + contents=[ + Content.from_computer_tool_call( + id=item["id"], + call_id=item["call_id"], + actions=actions, + status=item.get("status"), + pending_safety_checks=_computer_safety_checks(item.get("pending_safety_checks")), + additional_properties=props, + ) + ], + ) + + if item["type"] == "computer_call_output": + return Message( + role="tool", + contents=[ + Content.from_computer_tool_result( + id=item.get("id"), + call_id=item["call_id"], + screenshot=_computer_screenshot_from_output(item["output"]), + status=item.get("status"), + acknowledged_safety_checks=_computer_safety_checks(item.get("acknowledged_safety_checks")), + ) + ], + ) + + if item["type"] == "custom_tool_call": + return Message( + role="assistant", + contents=[ + Content.from_function_call( + item["call_id"], + item["name"], + arguments=item["input"], + informational_only=True, + hosted=True, + ) + ], + ) + + if item["type"] == "custom_tool_call_output": + output = _json_safe_to_str(item["output"]) + # Hosted-MCP results land here because the host writes them via + # `aoutput_item_custom_tool_call_output` (see `_OutputItemTracker.handle` for + # `mcp_server_tool_result`). The persisted `call_id` keeps its + # `mcp_*` prefix; on read, route those back to a hosted-MCP result + # Content so the chat-client serialize layer can coalesce them + # onto a single `mcp_call` input item with `output` populated. + # Issue #5546. + if item["call_id"] and item["call_id"].startswith("mcp_"): + return Message( + role="tool", + contents=[Content.from_mcp_server_tool_result(call_id=item["call_id"], output=output)], + ) + return Message( + role="tool", + contents=[Content.from_function_result(item["call_id"], result=output)], + ) + + if item["type"] == "apply_patch_call": + return Message( + role="assistant", + contents=[ + Content.from_function_call( + item["call_id"], + "apply_patch", + arguments=_json_safe_to_str(item["operation"]), + informational_only=True, + hosted=True, + ) + ], + ) + + if item["type"] == "apply_patch_call_output": + return Message( + role="tool", + contents=[Content.from_function_result(item["call_id"], result=_json_safe_to_str(item.get("output")))], + ) + + raise ValueError(f"Unsupported {_item_type_name} type: {item['type']}") + + +async def _output_items_to_messages( # pyright: ignore[reportUnusedFunction] + history: Sequence[OutputItem], +) -> list[Message]: + """Converts a sequence of OutputItem objects to a list of Message objects. + + Historical approval controls are not replayed. Native session state owns + pending requests; only current input carries new approval decisions. + + Args: + history (Sequence[OutputItem]): The sequence of OutputItem objects to convert. + + Returns: + list[Message]: The list of Message objects. + """ + messages: list[Message] = [] + for item in history: + if item["type"] in ("mcp_approval_request", "mcp_approval_response"): + continue + messages.append(await _output_item_to_message(item)) + return messages + + +def _approval_arguments_match(stored: Any, current: Any) -> bool: + """Compare wire arguments without coercing JSON value types or rounding numbers.""" + if not isinstance(stored, str): + return False + current_text = _json_safe_to_str(current) + if stored == current_text: + return True + try: + stored_value = json.loads(stored, parse_float=Decimal) + current_value = json.loads(current_text, parse_float=Decimal) + except json.JSONDecodeError: + return False + + def matches(left: Any, right: Any) -> bool: + if type(left) is not type(right): + return False + if isinstance(left, dict): + left_dict = cast(dict[str, Any], left) + right_dict = cast(dict[str, Any], right) + return left_dict.keys() == right_dict.keys() and all( + matches(value, right_dict[key]) for key, value in left_dict.items() + ) + if isinstance(left, list): + left_list = cast(list[Any], left) + right_list = cast(list[Any], right) + return len(left_list) == len(right_list) and all( + matches(left_item, right_item) for left_item, right_item in zip(left_list, right_list, strict=True) + ) + return bool(left == right) + + return matches(stored_value, current_value) + + +def _native_approval_id( # pyright: ignore[reportUnusedFunction] + item: Mapping[str, Any], *, session: AgentSession | None = None +) -> str: + """Resolve stored wire correlation without allowing it to redirect native authority.""" + raw_metadata = item.get("internal_metadata") + metadata: Mapping[str, Any] = cast(Mapping[str, Any], raw_metadata) if isinstance(raw_metadata, Mapping) else {} + request_id = metadata.get(_NATIVE_APPROVAL_ID_KEY) + if not isinstance(request_id, str) or not request_id: + raise _PublicResponseFailure( + ValueError("The approval uses an unsupported stored format; start a new conversation.") + ) + if session is not None: + if metadata.get(_NATIVE_APPROVAL_SESSION_KEY) != session.session_id: + raise _PublicResponseFailure(ValueError("The approval does not belong to this native session.")) + request = _load_pending_approval_requests(session).get(request_id) + function_call = request.function_call if request is not None else None + if ( + function_call is None + or item.get("server_label") != (function_call.server_name or "agent_framework") + or item.get("name") != function_call.name + or not _approval_arguments_match(item.get("arguments"), function_call.arguments) + ): + raise _PublicResponseFailure( + ValueError("The stored approval does not match the current native pending request.") + ) + return request_id + + +async def _output_item_to_message(item: OutputItem, *, session: AgentSession | None = None) -> Message: + """Converts an OutputItem to a Message. + + Args: + item (OutputItem): The OutputItem to convert. + session: Native session holding the agent's authoritative pending requests. + + Returns: + Message: The converted Message. + + Raises: + ValueError: If the OutputItem type is not supported. + """ + if item["type"] == "oauth_consent_request": + item_data = cast("Mapping[str, Any]", item) + server_label = item_data.get("server_label") + additional_properties = ( + { + "openai": {"server_label": server_label}, + "server_label": server_label, + } + if isinstance(server_label, str) and server_label + else None + ) + return Message( + role="assistant", + contents=[ + Content.from_oauth_consent_request( + item["consent_link"], + additional_properties=additional_properties, + ) + ], + ) + + if item["type"] == "structured_outputs": + return Message(role="assistant", contents=[Content.from_text(_json_safe_to_str(item["output"]))]) + + return await _item_to_message( + cast(Item, item), + session=session, + _item_type_name="OutputItem", + ) + + +def _convert_output_message_content(content: OutputMessageContent) -> Content: + """Converts an OutputMessageContent to a Content object. + + Args: + content (OutputMessageContent): The OutputMessageContent to convert. + + Returns: + Content: The converted Content object. + + Raises: + ValueError: If the OutputMessageContent type is not supported. + """ + if content["type"] == "output_text": + return Content.from_text(content["text"]) + if content["type"] == "refusal": + return Content.from_text( + content["refusal"], + additional_properties=_refusal_additional_properties(), + ) + + # Defensive: `OutputMessageContent` currently only supports `output_text` and `refusal`, + # but if new types are added in the future, this will catch them. + raise ValueError(f"Unsupported OutputMessageContent type: {content['type']}") + + +def _convert_file_data(data_uri: str, filename: str | None = None) -> Content: + """Convert a file_data data URI to a Content object. + + For text/* MIME types, decodes the base64 content and returns it as text. + For other types, returns a URI-based Content with the filename preserved. + """ + # Parse data URI: data:;base64, + if data_uri.startswith("data:") and ";base64," in data_uri: + header, encoded = data_uri.split(";base64,", 1) + media_type = header[len("data:") :] + if media_type.startswith("text/"): + try: + decoded_text = base64.b64decode(encoded).decode("utf-8") + except (ValueError, UnicodeDecodeError): + logger.warning( + "Failed to decode text/* file_data as UTF-8, falling through to URI passthrough.", + exc_info=True, + ) + else: + prefix = f"[File: {filename}]\n" if filename else "" + return Content.from_text(f"{prefix}{decoded_text}") + additional_properties = ( + { + "openai": {"content_type": "input_file"}, + "openai_content_type": "input_file", + "filename": filename, + } + if filename + else { + "openai": {"content_type": "input_file"}, + "openai_content_type": "input_file", + } + ) + return Content.from_uri(data_uri, name=filename, additional_properties=additional_properties) + + +def _convert_message_content(content: MessageContent) -> Content: + """Converts a MessageContent to a Content object. + + Args: + content (MessageContent): The MessageContent to convert. + + Returns: + Content: The converted Content object. + + Raises: + ValueError: If the MessageContent type is not supported. + """ + if content["type"] == "input_text": + return Content.from_text(content["text"]) + if content["type"] == "output_text": + return Content.from_text(content["text"]) + if content["type"] == "text": + return Content.from_text(content["text"]) + if content["type"] == "summary_text": + return Content.from_text(content["text"]) + if content["type"] == "refusal": + return Content.from_text( + content["refusal"], + additional_properties=_refusal_additional_properties(), + ) + if content["type"] == "reasoning_text": + return Content.from_text_reasoning(text=content["text"]) + if content["type"] == "input_image": + detail = content.get("detail") + additional_properties = { + "openai": { + "content_type": "input_image", + **({"image_detail": detail} if isinstance(detail, str) else {}), + }, + "openai_content_type": "input_image", + **({"detail": detail} if isinstance(detail, str) else {}), + } + if image_url := content.get("image_url"): + if image_url.startswith("data:"): + return Content.from_uri(image_url, additional_properties=additional_properties) + return Content.from_uri(image_url, media_type="image/*", additional_properties=additional_properties) + if file_id := content.get("file_id"): + return Content.from_hosted_file(file_id, additional_properties=additional_properties) + if content["type"] == "input_file": + filename = content.get("filename") + additional_properties = { + "openai": {"content_type": "input_file"}, + "openai_content_type": "input_file", + **({"filename": filename} if isinstance(filename, str) else {}), + } + if file_url := content.get("file_url"): + return Content.from_uri( + file_url, + name=filename if isinstance(filename, str) else None, + additional_properties=additional_properties, + ) + if file_id := content.get("file_id"): + return Content.from_hosted_file( + file_id, + name=filename if isinstance(filename, str) else None, + additional_properties=additional_properties, + ) + if file_data := content.get("file_data"): + return _convert_file_data(file_data, content.get("filename")) + if content["type"] == "computer_screenshot": + return _computer_screenshot_from_output(content) + + raise ValueError(f"Unsupported MessageContent type: {content['type']}") + + +# endregion + +# region Output Item Conversion + + +def _json_default(value: Any) -> Any: + if is_dataclass(value) and not isinstance(value, type): + return asdict(value) + to_dict = getattr(value, "to_dict", None) + if callable(to_dict): + return to_dict() + return str(value) + + +def _json_safe_to_str(value: Any | None) -> str: + """Convert an argument or result value to a JSON-safe string. + + Args: + value: The value to convert, which can be a string, JSON-like object, or None. + + Returns: + The value as a JSON string. + """ + if value is None: + return "" + if isinstance(value, str): + return value + try: + return json.dumps(value, default=_json_default) + except (TypeError, ValueError): + return json.dumps(str(value)) + + +def _image_generation_result(outputs: Any) -> str | None: + """Return the base64 image of an image generation tool result, as the Responses API expects. + + Only a non-empty, valid base64 string or the valid base64 payload of a base64 data URI qualifies. Other URIs + (https, file, blob), data URIs without the base64 marker and malformed payloads are not base64, so they give + None. + """ + if isinstance(outputs, str): + return outputs if _is_base64(outputs) else None + items: Sequence[Any] = [outputs] + if isinstance(outputs, Sequence) and not isinstance(outputs, (bytes, bytearray)): + items = cast(Sequence[Any], outputs) + for item in items: + if isinstance(item, Content) and item.uri and item.uri.startswith("data:"): + # RFC 2397: only the metadata before the first comma can carry the marker, as its last parameter. + metadata, _, payload = item.uri.partition(",") + if metadata.endswith(";base64") and _is_base64(payload): + return payload + return None + + +def _is_base64(value: str) -> bool: + """Check whether a string is non-empty, valid base64.""" + if not value: + return False + try: + base64.b64decode(value, validate=True) + except ValueError: + return False + return True + + +def _reasoning_encrypted_content(content: Content) -> str | None: + """Return the opaque reasoning payload used for stateless replay.""" + encrypted_content = content.protected_data or content.additional_properties.get("encrypted_content") + return encrypted_content if isinstance(encrypted_content, str) else None + + +def _reasoning_output_item( + *, + item_id: str, + summary_texts: Sequence[str], + encrypted_content: str | None, + status: Literal["in_progress", "completed"], +) -> OutputItemReasoningItem: + """Build a hosted reasoning item while retaining provider replay metadata.""" + return OutputItemReasoningItem({ + "type": "reasoning", + "id": item_id, + "summary": [{"type": "summary_text", "text": text} for text in summary_texts], + "encrypted_content": encrypted_content, + "status": status, + }) + + +def _mcp_mapping_text(output: Mapping[Any, Any]) -> str | None: + """Extract text only from a recognized MCP text-content mapping.""" + text = output.get("text") + if not isinstance(text, str): + return None + if output.get("type") == "text" or set(output) == {"text"}: + return text + return None + + +def _stringify_mcp_output(output: Any) -> str: + """Convert hosted MCP output payloads into the string shape expected by mcp_call.output.""" + if output is None: + return "" + if isinstance(output, str): + return output + if isinstance(output, Mapping): + mapping = cast(Mapping[Any, Any], output) + if (text := _mcp_mapping_text(mapping)) is not None: + return text + return _json_safe_to_str(mapping) + if isinstance(output, Sequence) and not isinstance(output, (str, bytes, bytearray)): + parts: list[str] = [] + entries = cast(Sequence[Any], output) + for entry in entries: + if isinstance(entry, str): + parts.append(entry) + continue + if isinstance(entry, Content) and entry.type == "text": + parts.append(entry.text or "") + continue + if isinstance(entry, Mapping) and (text := _mcp_mapping_text(cast(Mapping[Any, Any], entry))) is not None: + parts.append(text) + continue + return _json_safe_to_str(entries) + return "".join(parts) + return _json_safe_to_str(output) + + +# endregion diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_responses.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py similarity index 71% rename from python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_responses.py rename to python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py index 2df89e5ecb7..f63307c281b 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_responses.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses_workflow.py @@ -1,6 +1,6 @@ # Copyright (c) Microsoft. All rights reserved. -"""Native Responses workflow projection, with durable output preceding publication.""" +"""Host native workflows through Responses, with durable output preceding publication.""" from __future__ import annotations @@ -9,12 +9,12 @@ import json import logging import uuid -from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping +from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping, Sequence from contextlib import AbstractAsyncContextManager, AsyncExitStack, aclosing from contextvars import ContextVar from dataclasses import dataclass, fields, is_dataclass from datetime import date, datetime -from typing import Any, cast +from typing import Any, Literal, cast from agent_framework import ( AgentResponse, @@ -29,40 +29,52 @@ WorkflowCheckpointException, WorkflowEvent, ) +from agent_framework._telemetry import mark_feature_used from anyio import CancelScope -from azure.ai.agentserver.core import AgentConfig, get_request_context -from azure.ai.agentserver.responses import ResponseContext +from azure.ai.agentserver.core import get_request_context +from azure.ai.agentserver.responses import ResponseContext, ResponseProviderProtocol from azure.ai.agentserver.responses._id_generator import IdGenerator from azure.ai.agentserver.responses.aio import ResponseEventStream +from azure.ai.agentserver.responses.hosting import ResponsesAgentServerHost from azure.ai.agentserver.responses.models import CreateResponse, ResponseObject, ResponseStreamEvent from azure.ai.agentserver.responses.streaming._checkpoint import ResponseCheckpointEvent from pydantic import BaseModel from starlette.requests import Request from starlette.responses import Response, StreamingResponse -from starlette.routing import Route, Router +from starlette.routing import Route from starlette.types import Send -from ._request import ( - HostedResponseRequest, - OptionsHook, - WorkflowTurn, - prepare_response_options, - response_run_options, - validate_request_options, -) -from ._responses import ( +from ._feature_usage import FeatureIndex +from ._responses_shared import ( _HOSTED_PROVIDER_USAGE_KEY, # pyright: ignore[reportPrivateUsage] _agent_response_updates, # pyright: ignore[reportPrivateUsage] _hosted_internal_metadata, # pyright: ignore[reportPrivateUsage] _item_to_message, # pyright: ignore[reportPrivateUsage] + _native_approval_id, # pyright: ignore[reportPrivateUsage] _OutputItemTracker, # pyright: ignore[reportPrivateUsage] _request_without_reserved_metadata, # pyright: ignore[reportPrivateUsage] _SignalledIterator, # pyright: ignore[reportPrivateUsage] ) from ._scope import FoundryRequestScope -from ._state_store import CheckpointStoreProvider, ContextScopedStoreProvider -from ._workflow_source import WorkflowResolver, WorkflowSource, prepare_workflow_kwargs, workflow_agents -from ._workflow_state import HostedWorkflowRun, WorkflowConflictError +from ._shared import ( + CheckpointStoreProvider, + ContextScopedStoreProvider, + HostedWorkflowRun, + OptionsHook, + ResponsesRequest, + WorkflowConflictError, + WorkflowResolver, + WorkflowSource, + WorkflowTurn, + _build_response_server_options, # pyright: ignore[reportPrivateUsage] + prepare_response_options, + prepare_workflow_kwargs, + response_run_options, + validate_request_options, + validate_workflow_source, + workflow_agents, +) +from ._toolbox import _normalize_allowed_oauth_consent_origins # pyright: ignore[reportPrivateUsage] logger = logging.getLogger(__name__) @@ -150,7 +162,7 @@ def _snapshot(stream: ResponseEventStream) -> dict[str, Any]: return cast(dict[str, Any], json.loads(json.dumps(_wire_value(stream.response), allow_nan=False))) -async def _load_replies(request: HostedResponseRequest, run: HostedWorkflowRun) -> dict[str, Any]: +async def _load_replies(request: ResponsesRequest, run: HostedWorkflowRun) -> dict[str, Any]: items = await request.get_input_items() if not run.pending_requests or not items: raise _WorkflowRequestError("Workflow replies require an exact pending stored checkpoint.") @@ -216,36 +228,12 @@ async def _load_replies(request: HostedResponseRequest, run: HostedWorkflowRun) return dict(validated.responses or {}) -class _CheckpointApprovals: - """Collect wire aliases in the same per-response pair as the pending checkpoint.""" - - def __init__(self, checkpoint: WorkflowCheckpoint | None, aliases: dict[str, str]) -> None: - self.pending = checkpoint.pending_request_info_events if checkpoint is not None else {} - self.aliases = aliases - - async def save_approval_request(self, approval_request_id: str, request: Content) -> None: - matches = [ - request_id - for request_id, event in self.pending.items() - if isinstance(event.data, Content) and event.data.to_dict() == request.to_dict() - ] - if len(matches) != 1 or matches[0] in self.aliases.values(): - raise _WorkflowRequestError("Approval output must match one unique request in the exact pause checkpoint.") - self.aliases[approval_request_id] = matches[0] - - async def load_approval_request(self, approval_request_id: str) -> Content: - request_id = self.aliases.get(approval_request_id) - event = self.pending.get(request_id) if request_id is not None else None - if event is None or not isinstance(event.data, Content): - raise _WorkflowRequestError("The approval is unavailable in this pending checkpoint.") - return event.data - - async def _project( event: WorkflowEvent[Any], stream: ResponseEventStream, tracker: _OutputItemTracker, - approvals: _CheckpointApprovals, + checkpoint: WorkflowCheckpoint | None, + aliases: dict[str, str], ) -> list[ResponseStreamEvent]: updates: list[AgentResponseUpdate] = [] complete_message = False @@ -256,14 +244,14 @@ async def _project( elif isinstance(event.data, Content) and event.data.type in ("function_call", "computer_tool_call"): content = Content.from_dict(event.data.to_dict()) content.call_id = wire_id - approvals.aliases[wire_id] = event.request_id + aliases[wire_id] = event.request_id else: content = Content.from_function_call( wire_id, "request_info", arguments={"request_id": wire_id, "request": _wire_value(event.data)}, ) - approvals.aliases[wire_id] = event.request_id + aliases[wire_id] = event.request_id if isinstance(event.data, Content) and event.data.type == "oauth_consent_request": updates.append(AgentResponseUpdate(contents=[event.data])) updates.append(AgentResponseUpdate(contents=[content])) @@ -308,43 +296,140 @@ async def _project( return events events: list[ResponseStreamEvent] = [] for update in updates: - events.extend([ + projected = [ item async for item in tracker.handle_update( update, - approval_storage=approvals, complete_message=complete_message, ) - ]) + ] + for item_event in projected: + raw_item = item_event.get("item") + if item_event.get("type") != "response.output_item.added" or not isinstance(raw_item, Mapping): + continue + item = cast(Mapping[str, Any], raw_item) + if item.get("type") != "mcp_approval_request": + continue + native_id = _native_approval_id(item) + approval_content = next( + content + for content in update.contents + if content.type == "function_approval_request" and content.id == native_id + ) + pending = checkpoint.pending_request_info_events if checkpoint is not None else {} + matches = [ + request_id + for request_id, request in pending.items() + if isinstance(request.data, Content) + and request.data.type == "function_approval_request" + and request.data.to_dict() == approval_content.to_dict() + ] + wire_id = item.get("id") + if not isinstance(wire_id, str) or len(matches) != 1 or matches[0] in aliases.values(): + raise _WorkflowRequestError( + "Approval output must match one unique request in the exact pause checkpoint." + ) + aliases[wire_id] = matches[0] + events.extend(projected) return events -class NativeResponsesWorkflow: - """The native-only adapter; ordinary agents and legacy wrapper machinery remain separate.""" +class ResponsesWorkflowHostServer(ResponsesAgentServerHost): + """Host a native, checkpoint-owned workflow through the Responses protocol.""" def __init__( self, - source: WorkflowSource[HostedResponseRequest], - parser: Callable[[HostedResponseRequest], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], + workflow: WorkflowSource[ResponsesRequest], *, - config: AgentConfig, - checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage], - prepare_options: OptionsHook | None, - resilient_background: bool, - allowed_oauth_consent_origins: frozenset[str] | None, + parse_input: Callable[[ResponsesRequest], WorkflowTurn[Any] | Awaitable[WorkflowTurn[Any]]], + prefix: str = "", + response_store: ResponseProviderProtocol | None = None, + checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage] | None = None, + prepare_options: OptionsHook | None = None, + allowed_oauth_consent_origins: Sequence[str] | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + resilient_background: bool = False, + log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None, + agent_server_kwargs: Mapping[str, Any] | None = None, ) -> None: - self.resolver = WorkflowResolver(source) - self.parser = parser - self.config = config - self.checkpoint_store_provider = checkpoint_store_provider - self.options_hook = prepare_options - self.resilient_background = resilient_background - self.allowed_origins = allowed_oauth_consent_origins - - def bind_streaming_route(self, router: Router, *, prefix: str, keep_alive: bool) -> None: + """Create a native workflow host with explicit parsing and checkpoint ownership. + + Args: + workflow: Sync or async request-aware factory, called once per request (including recovery). + Build fresh workflows, executors, agents, clients, providers, and mutable tools inside the + factory. Keep workflow and executor IDs stable for checkpoint restoration. The host closes + context-managed agents returned by the factory; applications own other resource cleanup. + Construct tool objects inside the factory. A module-level `@tool` creates a shared wrapper, + which the ownership guard rejects even for a pure function. A function defined elsewhere can + be wrapped with `tool(function)` inside the factory; applications still own any captured state. + + Keyword Args: + parse_input: Sync or async parser of this turn's `ResponsesRequest`. Return a typed + `WorkflowTurn(input=...)` or a complete batch of validated pending `responses`. + prefix: URL prefix for Responses routes. + response_store: Backend for caller-facing Responses persistence, not the wire-level `store` flag. + checkpoint_store_provider: Scoped checkpoint storage. Defaults to `CheckpointStoreProvider`. + Native workflow history and pending replies are checkpoint-owned, not agent-session-owned. + prepare_options: Sync or async hook receiving the hosted request and a copy of caller model + options before parsing/execution. Return the replacement mapping; hosting validates it. + This does not enable inner provider background or service-owned history. + allowed_oauth_consent_origins: Optional exact HTTPS origins for consent links. Omission retains + safe-HTTPS validation; an empty sequence rejects all origins. + sse_keep_alive_interval_seconds: Positive streaming heartbeat interval in seconds. + `None` inherits a positive `SSE_KEEPALIVE_INTERVAL`, otherwise heartbeats are disabled. + Zero and negative explicit values are invalid. This is not a request or inference timeout. + shutdown_grace_period_seconds: Positive shutdown wait in seconds, defaulting to 10. + The SDK also uses it when DELETE waits for response persistence. Not a request timeout. + resilient_background: Permit checkpoint-backed outer workflow recovery, defaulting to + `False`. This does not enable inner provider background or guarantee exactly-once effects. + Framework-generated SDK options keep steering disabled. + log_level: Logging level forwarded unchanged to AgentServer. `None` preserves the SDK default. + agent_server_kwargs: Caller-owned settings expanded into the AgentServer SDK constructor. + Hosting does not validate their contents or mutate the mapping. SDK errors propagate. + + Raises: + TypeError: If the workflow is not a request-aware factory, the parser is not callable, or an + agent-only setting is supplied. + ValueError: If a supported host setting has an invalid value. + """ + validate_workflow_source(workflow) + if not callable(parse_input): + raise TypeError("parse_input must be a callable.") + if prepare_options is not None and not callable(prepare_options): + raise TypeError("prepare_options must be a callable.") + allowed_origins = _normalize_allowed_oauth_consent_origins(allowed_oauth_consent_origins) + options = _build_response_server_options( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=resilient_background, + ) + super().__init__( + prefix=prefix, + options=options, + store=response_store, + log_level=log_level, + **({} if agent_server_kwargs is None else agent_server_kwargs), + ) + self._resolver = WorkflowResolver(workflow) + self._parse_input = parse_input + self._checkpoint_store_provider = ( + CheckpointStoreProvider() if checkpoint_store_provider is None else checkpoint_store_provider + ) + self._prepare_options = prepare_options + self._resilient_background = resilient_background + self._allowed_origins = allowed_origins + self._bind_streaming_route( + prefix=prefix, + keep_alive=(sse_keep_alive_interval_seconds or self.config.sse_keepalive_interval) > 0, + ) + self.response_handler(self._handle_response) + mark_feature_used(FeatureIndex.FOUNDRY_HOSTING) + + def _bind_streaming_route(self, *, prefix: str, keep_alive: bool) -> None: """Adapt only the native POST route's public StreamingResponse cleanup lifetime.""" path = f"/{prefix.strip('/')}/responses" if prefix.strip("/") else "/responses" - for index, route in enumerate(router.routes): + for index, route in enumerate(self.router.routes): if not isinstance(route, Route) or route.path != path or "POST" not in (route.methods or ()): continue endpoint = cast(Callable[[Request], Awaitable[Response]], route.endpoint) @@ -357,7 +442,7 @@ async def native_endpoint( return _NativeWorkflowStreamingResponse(response, keep_alive=keep_alive) return response - router.routes[index] = Route( + self.router.routes[index] = Route( route.path, endpoint=native_endpoint, methods=list(route.methods or ()), @@ -367,7 +452,7 @@ async def native_endpoint( return raise RuntimeError("The native Responses streaming route is unavailable.") - async def response_events( + async def _handle_response( self, request: CreateResponse, context: ResponseContext, @@ -375,7 +460,7 @@ async def response_events( ) -> AsyncGenerator[ResponseStreamEvent | ResponseCheckpointEvent]: """Keep factory, resources, and native generator cleanup in one persistent task.""" lifetime = _NATIVE_STREAM_LIFETIME.get() - inner = self.handle_response(request, context, cancellation_signal) + inner = self._run_workflow(request, context, cancellation_signal) if lifetime is None or request.get("background") is True: async with aclosing(inner): async for event in inner: @@ -387,7 +472,7 @@ async def response_events( async for event in events: yield event - async def handle_response( + async def _run_workflow( self, request: CreateResponse, context: ResponseContext, @@ -427,26 +512,20 @@ async def abort() -> bool: scope = FoundryRequestScope.from_context( self.config, platform_context, local_session_id=context.response_id ) - hosted = HostedResponseRequest(request, context, scope, response_run_options(request)) - await prepare_response_options(hosted, self.options_hook) + hosted = ResponsesRequest(request, context, scope, response_run_options(request)) + await prepare_response_options(hosted, self._prepare_options) validate_request_options(hosted.options) - workflow = await self.resolver.resolve(hosted) + workflow = await self._resolver.resolve(hosted) resources = AsyncExitStack() await resources.__aenter__() for agent in workflow_agents(workflow): if isinstance(agent, AbstractAsyncContextManager): await resources.enter_async_context(agent) resilient = ( - self.resilient_background and request.get("background") is True and request.get("store") is not False + self._resilient_background and request.get("background") is True and request.get("store") is not False ) if context.is_recovery and not resilient: raise _WorkflowRequestError("Native workflow recovery requires a stored resilient background response.") - if not self.resolver.is_factory and ( - context.is_recovery - or context.conversation_id is not None - or request.get("previous_response_id") is not None - ): - raise _WorkflowRequestError("Native workflow continuation requires a fresh request-aware factory.") phase = "scoped continuation" run = await HostedWorkflowRun.prepare( workflow, @@ -454,12 +533,12 @@ async def abort() -> bool: response_id=context.response_id, config=self.config, platform_context=platform_context, - checkpoint_store_provider=self.checkpoint_store_provider, + checkpoint_store_provider=self._checkpoint_store_provider, previous_response_id=request.get("previous_response_id"), conversation_id=context.conversation_id, stored=request.get("store") is not False, recovery=context.is_recovery, - fresh_factory=self.resolver.is_factory, + fresh_factory=True, ) if context.is_recovery and run.snapshot is not None: if run.snapshot.get("id") != context.response_id: @@ -471,7 +550,7 @@ async def abort() -> bool: yield wire(stream.emit_in_progress()) hosted._set_workflow_responses(lambda: _load_replies(hosted, run)) # pyright: ignore[reportPrivateUsage] phase = "input parsing" - turn = run.recovery_turn() if context.is_recovery else self.parser(hosted) + turn = run.recovery_turn() if context.is_recovery else self._parse_input(hosted) if inspect.isawaitable(turn): turn = await turn phase = "input validation" @@ -479,10 +558,10 @@ async def abort() -> bool: if ( run.stored and turn.input is not None - and isinstance(self.checkpoint_store_provider, CheckpointStoreProvider) + and isinstance(self._checkpoint_store_provider, CheckpointStoreProvider) ): try: - self.checkpoint_store_provider.validate_checkpoint_value(turn.input) + self._checkpoint_store_provider.validate_checkpoint_value(turn.input) except WorkflowCheckpointException as exc: raise _WorkflowRequestError( "Stored native workflow input uses an application type that is not checkpoint-allowlisted. " @@ -494,7 +573,7 @@ async def abort() -> bool: scope, options=hosted.options, stored=run.stored, - fresh_factory=self.resolver.is_factory, + fresh_factory=True, ) if cancellation_signal.is_set(): return @@ -502,7 +581,7 @@ async def abort() -> bool: aliases = dict(run.approvals) if context.is_recovery else {} represented_pending: set[str] = set(run.pending_requests) if context.is_recovery else set() buffered: list[WorkflowEvent[Any]] = [] - tracker = _OutputItemTracker(stream, self.allowed_origins) + tracker = _OutputItemTracker(stream, self._allowed_origins) async def flush(checkpoint_id: str | None) -> list[ResponseStreamEvent]: nonlocal stream, tracker, aliases @@ -514,11 +593,10 @@ async def flush(checkpoint_id: str | None) -> list[ResponseStreamEvent]: ) draft.emit_created() draft.emit_in_progress() - next_tracker = _OutputItemTracker(draft, self.allowed_origins) - approval_projection = _CheckpointApprovals(checkpoint, next_aliases) + next_tracker = _OutputItemTracker(draft, self._allowed_origins) events: list[ResponseStreamEvent] = [] for event in buffered: - events.extend(await _project(event, draft, next_tracker, approval_projection)) + events.extend(await _project(event, draft, next_tracker, checkpoint, next_aliases)) events.extend(next_tracker.close()) _hosted_internal_metadata(draft)[_HOSTED_PROVIDER_USAGE_KEY] = next_tracker.usage_details await run.stage(_snapshot(draft), approvals=next_aliases, checkpoint_id=checkpoint_id) @@ -539,13 +617,9 @@ async def checkpoint_stamp() -> str | None: # ruff: ignore[unused-async] ) async with aclosing(iterator): async for event in iterator: - if event.type == "request_info" and event.request_id not in represented_pending: - if not self.resolver.is_factory: - raise _WorkflowRequestError( - "A pausing native workflow requires a fresh request-aware factory." - ) - buffered.append(event) - elif event.type in ("output", "intermediate", "data"): + if ( + event.type == "request_info" and event.request_id not in represented_pending + ) or event.type in ("output", "intermediate", "data"): buffered.append(event) elif event.type == "superstep_started" and run.stored and run.snapshot is None: await run.stage(_snapshot(stream), checkpoint_id=iterator.stamp) diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py new file mode 100644 index 00000000000..b162b9e2a07 --- /dev/null +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_shared.py @@ -0,0 +1,2090 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Share request contracts, options, factories, and native-state handling across Foundry hosts.""" + +from __future__ import annotations + +import hashlib +import inspect +import json +import logging +import secrets +import weakref +from abc import ABC, abstractmethod +from collections import OrderedDict +from collections.abc import AsyncGenerator, Awaitable, Callable, Iterator, Mapping, Sequence +from copy import deepcopy +from dataclasses import asdict, dataclass, field, replace +from datetime import datetime +from types import MappingProxyType +from typing import Any, Generic, Literal, Protocol, TypeAlias, TypeGuard, cast + +from agent_framework import ( + Agent, + AgentContext, + AgentExecutor, + AgentMiddleware, + AgentResponse, + AgentResponseUpdate, + AgentRunInputs, + AgentSession, + ChatResponse, + ChatResponseUpdate, + CheckpointID, + CheckpointStorage, + Content, + HistoryProvider, + InMemoryHistoryProvider, + Message, + RawAgent, + SessionStore, + SupportsAgentRun, + Workflow, + WorkflowAgent, + WorkflowCheckpoint, + WorkflowCheckpointException, + WorkflowEvent, + WorkflowExecutor, + WorkflowInvocationKwargs, + WorkflowRunState, +) +from agent_framework._workflows._agent_executor import ( + _validate_computer_tool_result, # pyright: ignore[reportPrivateUsage] +) +from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value +from agent_framework._workflows._const import ( + RESOLVED_WORKFLOW_RUN_KWARGS_KEY, + ROUTED_WORKFLOW_RUN_KWARGS_KEY, + WORKFLOW_RUN_KWARGS_KEY, +) +from agent_framework._workflows._typing_utils import is_instance_of +from agent_framework._workflows._workflow import ( + _coerce_request_info_response, # pyright: ignore[reportPrivateUsage] +) +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext +from azure.ai.agentserver.core.storage import ( + FoundryStateStore, + FoundryStorageConflictError, + FoundryStoragePreconditionError, +) +from azure.ai.agentserver.responses import ResponseContext, ResponsesServerOptions +from azure.ai.agentserver.responses.models import CreateResponse, Item +from typing_extensions import TypeVar + +from ._scope import FoundryRequestScope + +UnsupportedOptions: TypeAlias = Literal["ignore", "warn", "error"] + + +def validate_sse_keep_alive_interval(value: int | None) -> None: + """Validate an explicit heartbeat interval without replacing SDK environment resolution.""" + if value is not None: + if isinstance(value, bool) or not isinstance(value, int): + raise TypeError("sse_keep_alive_interval_seconds must be an integer or None.") + if value <= 0: + raise ValueError("sse_keep_alive_interval_seconds must be positive when supplied.") + + +def _build_response_server_options( # pyright: ignore[reportUnusedFunction] + *, + max_history_items: int | None = None, + sse_keep_alive_interval_seconds: int | None = None, + shutdown_grace_period_seconds: int = 10, + resilient_background: bool = False, +) -> ResponsesServerOptions: + """Build supported SDK settings, leaving an omitted history limit at its SDK default.""" + validate_sse_keep_alive_interval(sse_keep_alive_interval_seconds) + if isinstance(shutdown_grace_period_seconds, bool) or not isinstance(shutdown_grace_period_seconds, int): + raise TypeError("shutdown_grace_period_seconds must be an integer.") + if shutdown_grace_period_seconds <= 0: + raise ValueError("shutdown_grace_period_seconds must be positive.") + if not isinstance(resilient_background, bool): + raise TypeError("resilient_background must be a boolean.") + history_options: dict[str, Any] = {} + if max_history_items is not None: + if isinstance(max_history_items, bool) or not isinstance(max_history_items, int): + raise TypeError("max_history_items must be an integer or None.") + if max_history_items <= 0: + raise ValueError("max_history_items must be positive when supplied.") + history_options["default_fetch_history_count"] = max_history_items + return ResponsesServerOptions( + sse_keep_alive_interval_seconds=sse_keep_alive_interval_seconds, + shutdown_grace_period_seconds=shutdown_grace_period_seconds, + resilient_background=resilient_background, + steerable_conversations=False, + **history_options, + ) + + +InputT = TypeVar("InputT", default=Any) + + +@dataclass(frozen=True) +class WorkflowTurn(Generic[InputT]): + """A typed workflow start input or a complete batch of pending replies. + + Args: + input: Application input accepted by the start executor. ``None`` is not a start input. + responses: Replies keyed by the exact pending workflow request IDs. The host validates + the batch against its scoped checkpoint before consuming any reply authority. + client_kwargs: Existing workflow kwargs forwarded to agent clients, not raw workflow run arguments. + function_invocation_kwargs: Existing workflow kwargs forwarded to tools. Trusted hosting + identity is supplied separately and cannot be overridden. + stream: Application streaming intent for protocols with a custom request parser. + Responses continues to use its native request's ``stream`` flag. + """ + + input: InputT | None = None + responses: Mapping[str, Any] | None = None + client_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None + function_invocation_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None + stream: bool = False + + def __post_init__(self) -> None: + if (self.input is None and self.responses is None) or (self.input is not None and self.responses is not None): + raise ValueError("WorkflowTurn requires exactly one of input or non-empty responses.") + if self.responses is not None: + if ( + not isinstance(self.responses, Mapping) + or not self.responses + or any(not isinstance(key, str) or not key for key in self.responses) + ): + raise ValueError("WorkflowTurn.responses must be a non-empty mapping of request IDs to replies.") + object.__setattr__(self, "responses", MappingProxyType(dict(self.responses))) + for name in ("client_kwargs", "function_invocation_kwargs"): + value = getattr(self, name) + if value is not None and not isinstance(value, (Mapping, WorkflowInvocationKwargs)): + raise TypeError(f"WorkflowTurn.{name} must be a mapping or WorkflowInvocationKwargs.") + if not isinstance(self.stream, bool): + raise TypeError("WorkflowTurn.stream must be a boolean.") + + +@dataclass(frozen=True) +class InvocationRun: + """Messages, per-turn options, and streaming intent parsed from an Invocations request. + + Args: + messages: MAF message input for this turn, including typed `Message` or `Content` values. + options: Caller generation options for this turn, separate from agent defaults. + stream: Whether to stream framed SSE events rather than return a JSON response. + + Raises: + TypeError: If the messages, options, or streaming intent have invalid types. + """ + + messages: AgentRunInputs + options: Mapping[str, Any] = field(default_factory=lambda: dict[str, Any]()) + stream: bool = False + + def __post_init__(self) -> None: + if not ( + isinstance(self.messages, (str, Content, Message)) + or ( + isinstance(self.messages, Sequence) + and all(isinstance(message, (str, Content, Message)) for message in self.messages) + ) + ): + raise TypeError("InvocationRun.messages must be a string, Content, Message, or a sequence of them.") + if not isinstance(self.options, Mapping) or any(not isinstance(key, str) for key in self.options): + raise TypeError("InvocationRun.options must be a mapping with string keys.") + if not isinstance(self.stream, bool): + raise TypeError("InvocationRun.stream must be a boolean.") + + +_HOST_CONTROLLED_FIELDS = frozenset({ + "agent", + "agent_reference", + "agent_session_id", + "background", + "call_id", + "continuation_token", + "conversation", + "conversation_id", + "extra_body", + "input", + "previous_response_id", + "response_id", + "service_session_id", + "session_id", + "store", + "stream", + "user", + "user_id", +}) + + +_OPTION_NAMES = {"max_output_tokens": "max_tokens", "parallel_tool_calls": "allow_multiple_tool_calls"} + + +_NATIVE_FIELDS = frozenset(CreateResponse.__annotations__) + + +def response_run_options(request: CreateResponse) -> dict[str, Any]: + """Translate native generation fields, with flattened extra-body values winning collisions.""" + native: dict[str, Any] = {} + extra: dict[str, Any] = {} + nested_extra: dict[str, Any] = {} + for name, value in request.items(): + if value is None or (name == "model" and value == ""): + continue + if name == "extra_body": + if not isinstance(value, Mapping) or any( + not isinstance(key, str) for key in cast(Mapping[object, object], value) + ): + raise TypeError("extra_body must be a mapping of model options.") + if "extra_body" in value: + raise ValueError("Nested extra_body is not supported; flatten provider options instead.") + nested_extra.update(cast(Mapping[str, Any], value)) + elif name in _HOST_CONTROLLED_FIELDS: + continue + elif name in _NATIVE_FIELDS: + native[_OPTION_NAMES.get(name, name)] = value + else: + extra[name] = value + return { + **native, + **{name: value for name, value in extra.items() if name not in _HOST_CONTROLLED_FIELDS}, + **{name: value for name, value in nested_extra.items() if name not in _HOST_CONTROLLED_FIELDS}, + } + + +class ResponsesRequest: + """Expose a copied request, trusted scope, and this turn's input to factories, parsers, and option hooks.""" + + def __init__( + self, + request: CreateResponse, + context: ResponseContext, + scope: FoundryRequestScope, + options: Mapping[str, Any], + ) -> None: + self.request: Mapping[str, Any] = MappingProxyType(deepcopy(dict(request))) + self.scope = scope + self.response_id = context.response_id + self.conversation_id = context.conversation_id + self._context = context + self._options: Mapping[str, Any] = MappingProxyType(dict(options)) + self._workflow_responses: Callable[[], Awaitable[dict[str, Any]]] | None = None + + @property + def options(self) -> Mapping[str, Any]: + """Return this turn's effective caller options after the hook.""" + return self._options + + def set_options(self, options: Mapping[str, Any]) -> None: + """Replace the caller options without changing the agent's defaults.""" + self._options = MappingProxyType(dict(options)) + + async def get_input_items(self) -> list[Item]: + """Read only this turn's input items, not earlier conversation history.""" + return list(await self._context.get_input_items()) + + async def get_input_text(self) -> str | None: + """Read this turn's text input, if present.""" + return await self._context.get_input_text() + + async def get_workflow_responses(self) -> dict[str, Any]: + """Decode replies against this turn's exact scoped workflow checkpoint. + + Use this explicitly in ``parse_input`` to resume a native workflow. + Unknown, duplicate, stale, and incomplete reply batches are rejected before + the host claims the workflow or executes any response handler. + """ + if self._workflow_responses is None: + raise RuntimeError("Workflow replies require a native workflow host and a pending stored checkpoint.") + return await self._workflow_responses() + + def _set_workflow_responses(self, loader: Callable[[], Awaitable[dict[str, Any]]]) -> None: + self._workflow_responses = loader + + +OptionsHook: TypeAlias = Callable[ + [ResponsesRequest, dict[str, Any]], + Mapping[str, Any] | Awaitable[Mapping[str, Any]], +] + + +async def prepare_response_options(request: ResponsesRequest, hook: OptionsHook | None) -> None: + """Apply a synchronous or asynchronous developer hook to a copy of caller options.""" + if hook is None: + return + result = hook(request, dict(request.options)) + if inspect.isawaitable(result): + result = await result + if not isinstance(result, Mapping): + raise TypeError("prepare_options must return a mapping of MAF run options.") + request.set_options(result) + + +async def response_input_messages(request: ResponsesRequest) -> list[Message]: + """Convert only this Responses turn's input items to the legacy workflow message contract. + + This explicit migration helper does not load outer history, restore checkpoints, + or decode pending native workflow replies. Native workflows should prefer typed + application inputs and use ``request.get_workflow_responses()`` for resumable pauses. + """ + from ._responses_shared import _items_to_messages # pyright: ignore[reportPrivateUsage] + + return await _items_to_messages(await request.get_input_items()) + + +def validate_request_options(options: Mapping[str, Any]) -> None: + """Keep hosting identity, storage decisions, and private continuation out of model options.""" + reserved = _HOST_CONTROLLED_FIELDS.intersection(options) + if reserved: + raise ValueError(f"prepare_options cannot set host-controlled fields: {', '.join(sorted(reserved))}.") + + +def validate_default_transport_options(defaults: Mapping[str, Any], *, allow_agent_store: bool) -> None: + """Reject transport overrides that would bypass the host's inner storage and identity decisions.""" + extra_body = defaults.get("extra_body") + if extra_body is None: + return + if not isinstance(extra_body, Mapping) or any( + not isinstance(key, str) for key in cast(Mapping[object, object], extra_body) + ): + raise TypeError("Agent default extra_body must be a mapping of model options.") + reserved = _HOST_CONTROLLED_FIELDS.intersection(cast(Mapping[str, Any], extra_body)) + if allow_agent_store: + reserved -= {"store"} + if reserved: + raise ValueError( + "Agent default extra_body cannot set host-controlled fields: " + f"{', '.join(sorted(reserved))}. Use explicit agent defaults or history_source='inference_service'." + ) + + +def validate_unsupported_options(mode: str) -> UnsupportedOptions: + """Reject misspelled unsupported-options policies at host construction.""" + if mode not in ("ignore", "warn", "error"): + raise ValueError("unsupported_options must be 'ignore', 'warn', or 'error'.") + return mode + + +RequestT = TypeVar("RequestT") +AgentRequestT = TypeVar("AgentRequestT", contravariant=True) + + +class AgentFactory(Protocol[AgentRequestT]): + """Create a request-owned agent from the protocol's request context.""" + + def __call__(self, *, request: AgentRequestT) -> SupportsAgentRun | Awaitable[SupportsAgentRun]: + """Create an agent, optionally using the current request.""" + ... + + +AgentSource: TypeAlias = SupportsAgentRun | AgentFactory[RequestT] + + +def is_agent(value: object) -> TypeGuard[SupportsAgentRun]: + return not inspect.isclass(value) and isinstance(value, SupportsAgentRun) + + +def validate_agent_source(source: object) -> None: + if is_agent(source): + _validate_agent_type(source) + return + if not callable(source): + raise TypeError("agent must be an agent instance or a callable that creates one.") + _validate_agent_factory(source) + + +def _validate_agent_type(agent: SupportsAgentRun) -> None: + if isinstance(agent, WorkflowAgent): + raise TypeError( + "Agent hosts do not support WorkflowAgent. Use ResponsesWorkflowHostServer or " + "InvocationsWorkflowHostServer with a factory returning a native Workflow." + ) + + +def _validate_agent_factory(factory: Callable[..., object]) -> None: + message = ( + "agent factory must declare a required named 'request' parameter and accept " + "request=request with no other required arguments." + ) + try: + signature = inspect.signature(factory) + except (TypeError, ValueError) as exc: + raise TypeError(message) from exc + parameter = signature.parameters.get("request") + if ( + parameter is None + or parameter.kind not in (inspect.Parameter.POSITIONAL_OR_KEYWORD, inspect.Parameter.KEYWORD_ONLY) + or parameter.default is not inspect.Parameter.empty + ): + raise TypeError(message) + try: + signature.bind(request=object()) + except TypeError as exc: + raise TypeError(message) from exc + + +async def resolve_agent(source: AgentSource[RequestT], request: RequestT) -> SupportsAgentRun: + """Resolve an agent instance or request-scoped agent factory.""" + if is_agent(source): + _validate_agent_type(source) + return source + + factory = cast(AgentFactory[RequestT], source) + result = factory(request=request) + agent = await result if inspect.isawaitable(result) else result + if not is_agent(agent): + raise TypeError("The agent factory must return an object implementing SupportsAgentRun.") + _validate_agent_type(agent) + return agent + + +WorkflowSource: TypeAlias = Callable[[RequestT], Workflow | Awaitable[Workflow]] + + +_MAX_STRONG_RESOURCE_IDENTITIES = 1024 + + +_CLIENT_CONTROLS = frozenset({ + "additional_function_arguments", + "agent", + "background", + "call_id", + "checkpoint_id", + "checkpoint_storage", + "client_kwargs", + "continuation_token", + "conversation", + "conversation_id", + "extra_body", + "function_invocation_kwargs", + "fresh_factory", + "input", + "instructions", + "messages", + "middleware", + "previous_response_id", + "response_id", + "responses", + "service_session_id", + "session", + "session_id", + "agent_session_id", + "store", + "stream", + "tokenizer", + "tools", + "user", + "user_id", +}) + + +class _NativeAgentOptions(AgentMiddleware): + """Apply native workflow options at the existing agent boundary, not as client **kwargs.""" + + def __init__(self, scope: FoundryRequestScope) -> None: + self.scope = scope + + async def process(self, context: AgentContext, call_next: Callable[[], Awaitable[None]]) -> None: + options = context.client_kwargs.pop("options", {}) + if not isinstance(options, Mapping): + raise TypeError("Native workflow model options must be a mapping.") + options = cast(Mapping[str, Any], options) + if (_CLIENT_CONTROLS - {"store"}).intersection(options): + raise ValueError("Native workflow model options cannot override hosting or transport controls.") + if context.session is not None and context.session.service_session_id is not None: + raise RuntimeError("Native workflow checkpoints cannot resume a private downstream service session.") + effective = {**(context.options or {}), **options} + client = getattr(context.agent, "client", None) + if getattr(client, "STORES_BY_DEFAULT", None) is True: + effective["store"] = False + else: + if effective.pop("store", None) is True: + raise RuntimeError("The native workflow client cannot safely override a storing runtime option.") + context.options = effective + context.client_kwargs["additional_function_arguments"] = { + "session_id": self.scope.session_id, + "user_id": self.scope.user_id, + "call_id": self.scope.call_id, + } + + def check_result(result: AgentResponse[Any]) -> None: + if context.session is not None and context.session.service_session_id is not None: + raise RuntimeError("The native workflow client stored a service session despite store=False.") + if result.continuation_token is not None: + raise RuntimeError("Native workflows cannot checkpoint an unfinished provider background response.") + + if context.stream: + context.stream_result_hooks.append(check_result) + await call_next() + if not context.stream and isinstance(context.result, AgentResponse): + check_result(context.result) + + +def prepare_workflow_kwargs( + workflow: Workflow, + turn: WorkflowTurn[Any], + scope: FoundryRequestScope, + *, + options: Mapping[str, Any] | None = None, + stored: bool = True, + fresh_factory: bool = False, +) -> tuple[WorkflowInvocationKwargs, WorkflowInvocationKwargs]: + """Separate model options, explicit tool kwargs, and trusted current-call context. + + Agent executors use MAF checkpoint history, not downstream service storage. + Applications keep tool context in ``function_invocation_kwargs``; generation + options cannot become workflow, agent lifecycle, or storage controls. + ``Agent`` uses its existing middleware boundary without changing defaults. + Bare ``RawAgent`` requires ``fresh_factory`` from a validated resolver, + explicit non-storing defaults, and pre-materialized matching model overrides. + No nested options keyword is forwarded to a bare client's ``**kwargs``. + """ + client, _ = workflow._resolve_invocation_kwargs( # pyright: ignore[reportPrivateUsage] + turn.client_kwargs or {}, "client_kwargs" + ) + functions, _ = workflow._resolve_invocation_kwargs( # pyright: ignore[reportPrivateUsage] + turn.function_invocation_kwargs or {}, "function_invocation_kwargs" + ) + + def validate_values(value: Any) -> dict[str, Any]: + if not isinstance(value, Mapping) or any(not isinstance(key, str) for key in cast(Mapping[object, Any], value)): + raise TypeError("Workflow kwargs must have string-keyed mappings for global and executor values.") + return dict(cast(Mapping[str, Any], value)) + + global_client = validate_values(client.get("global_kwargs", {})) + specific_client = { + key: validate_values(value) for key, value in cast(dict[str, Any], client.get("executor_kwargs", {})).items() + } + for values in (global_client, *specific_client.values()): + if _CLIENT_CONTROLS.intersection(values): + raise ValueError("Workflow client kwargs cannot override hosting, agent, or transport controls.") + if "options" in values: + values["options"] = validate_values(values["options"]) + if _CLIENT_CONTROLS.intersection(values["options"]): + raise ValueError("Workflow model options cannot override hosting, agent, or transport controls.") + model_options = validate_values(options or {}) + if _CLIENT_CONTROLS.intersection(model_options): + raise ValueError("Workflow model options cannot override hosting, agent, or transport controls.") + + identity = {"session_id": scope.session_id, "user_id": scope.user_id, "call_id": scope.call_id} + global_client["additional_function_arguments"] = identity + global_model_options = global_client.pop("options", {}) + agents = list(workflow_agent_targets(workflow)) + for executor_id, agent_value in agents: + if not isinstance(agent_value, RawAgent): + raise TypeError( + "Native workflow custom agents must expose a verifiable client/storage contract. " + "Use Agent or a validated fresh RawAgent factory." + ) + agent = cast(RawAgent[Any], agent_value) + validate_default_transport_options(agent.default_options, allow_agent_store=False) + if any( + agent.default_options.get(name) is not None + for name in _CLIENT_CONTROLS - {"store", "instructions", "tools"} + ): + raise ValueError("Native workflow agents cannot have downstream continuation or hosting identity defaults.") + stores = getattr(agent.client, "STORES_BY_DEFAULT", None) + if not isinstance(stores, bool) or (not stores and agent.default_options.get("store") is True): + raise TypeError( + "Native workflow clients must declare storage support so hosting can disable inner storage." + ) + if not stored and any( + isinstance(provider, HistoryProvider) + and not isinstance(provider, InMemoryHistoryProvider) + and (provider.store_inputs or provider.store_outputs or provider.store_context_messages) + for provider in agent.context_providers + ): + raise ValueError("store=false cannot disable an external workflow HistoryProvider.") + values = specific_client.setdefault(executor_id, {}) + effective = {**global_model_options, **values.get("options", {}), **model_options} + values["additional_function_arguments"] = identity + if isinstance(agent_value, Agent): + if stores: + effective["store"] = False + values["options"] = effective + policies = [ + middleware for middleware in agent_value.middleware or () if isinstance(middleware, _NativeAgentOptions) + ] + if policies: + if len(policies) != 1 or policies[0].scope != scope: + raise RuntimeError("Native workflow option policy cannot be shared across request scopes.") + else: + agent_value.middleware = [*(agent_value.middleware or ()), _NativeAgentOptions(scope)] + else: + if not fresh_factory: + raise TypeError("Native RawAgent workflows require a validated request-aware fresh factory.") + if stores and agent.default_options.get("store") is not False: + raise ValueError("A storing native RawAgent client requires explicit factory default store=False.") + if any(agent.default_options.get(key) != value for key, value in effective.items()): + raise ValueError( + "RawAgent workflow generation overrides must be materialized in the fresh factory defaults." + ) + values.pop("options", None) + global_functions = validate_values(functions.get("global_kwargs", {})) + specific_functions = { + key: validate_values(value) for key, value in cast(dict[str, Any], functions.get("executor_kwargs", {})).items() + } + return ( + WorkflowInvocationKwargs(global_client, specific_client), + WorkflowInvocationKwargs(global_functions, specific_functions), + ) + + +def workflow_executors(workflow: Workflow) -> Iterator[AgentExecutor]: + """Walk agent executors, including those in built subworkflows.""" + for executor in workflow.get_executors_list(): + if isinstance(executor, AgentExecutor): + yield executor + elif isinstance(executor, WorkflowExecutor): + yield from workflow_executors(executor.workflow) + + +def workflow_agent_targets(workflow: Workflow) -> Iterator[tuple[str, SupportsAgentRun]]: + """Walk core and registry-backed executor agents with their run-kwargs routing IDs.""" + seen: set[tuple[str, int]] = set() + for graph in _workflow_graphs(workflow): + for executor in graph.get_executors_list(): + values: list[object] = [] + if isinstance(executor, AgentExecutor): + values.append(executor.agent) + registry = getattr(executor, "_agents", None) + if isinstance(registry, Mapping): + values.extend(cast(Mapping[object, object], registry).values()) + for value in values: + agent = value.agent if isinstance(value, AgentExecutor) else value + if not isinstance(agent, SupportsAgentRun): + raise TypeError("A workflow agent registry contains an unsupported agent value.") + key = (executor.id, id(agent)) + if key in seen: + continue + seen.add(key) + yield executor.id, agent + + +def workflow_agents(workflow: Workflow) -> Iterator[SupportsAgentRun]: + """Walk every unique agent owned by core or registry-backed executors.""" + seen: set[int] = set() + for _, agent in workflow_agent_targets(workflow): + if id(agent) in seen: + continue + seen.add(id(agent)) + yield agent + + +def _workflow_graphs(workflow: Workflow) -> Iterator[Workflow]: + yield workflow + for executor in workflow.get_executors_list(): + if isinstance(executor, WorkflowExecutor): + yield from _workflow_graphs(executor.workflow) + + +def validate_workflow_provider_state(workflow: Workflow, checkpoint: WorkflowCheckpoint | None = None) -> None: + """Refuse unexpected private service continuation before pairing or publishing native output.""" + if checkpoint is None: + for executor in workflow_executors(workflow): + if executor._session.service_session_id is not None: # pyright: ignore[reportPrivateUsage] + raise RuntimeError("The native workflow client stored a service session despite store=False.") + return + states = checkpoint.state.get("_executor_state", {}) + if not isinstance(states, Mapping): + raise ValueError("Invalid native workflow executor checkpoint state.") + states = cast(Mapping[str, Any], states) + for executor in workflow.get_executors_list(): + state = states.get(executor.id, {}) + if not isinstance(state, Mapping): + raise ValueError("Invalid native workflow executor checkpoint state.") + state = cast(Mapping[str, Any], state) + if isinstance(executor, AgentExecutor): + session = state.get("agent_session", {}) + if not isinstance(session, Mapping): + raise ValueError("Invalid native workflow agent checkpoint state.") + session = cast(Mapping[str, Any], session) + if session.get("service_session_id") is not None: + raise RuntimeError("The native workflow checkpoint contains unexpected private service continuation.") + elif isinstance(executor, WorkflowExecutor): + child = state.get("sub_workflow_checkpoint") + if child is not None: + if not isinstance(child, WorkflowCheckpoint): + raise ValueError("Invalid native subworkflow checkpoint state.") + validate_workflow_provider_state(executor.workflow, child) + + +def validate_workflow_source(source: object) -> None: + """Require a factory accepting the protocol's request object.""" + if isinstance(source, (Workflow, SupportsAgentRun)) or not callable(source) or inspect.isclass(source): + raise TypeError("workflow must be a request-aware factory returning a freshly built Workflow.") + try: + inspect.signature(source).bind(object()) + except (TypeError, ValueError) as exc: + raise TypeError("A workflow factory must accept one request argument.") from exc + + +class WorkflowResolver(Generic[RequestT]): + """Reject reuse of mutable graphs and known request-bound resources. + + This is an ownership guard, not a clone operation or a proof that arbitrary + application globals are stateless. Factories must allocate their resources + without performing workflow or tool side effects. + """ + + def __init__(self, source: WorkflowSource[RequestT]) -> None: + validate_workflow_source(source) + self.source = source + self._owned: dict[int, weakref.ReferenceType[Any]] = {} + self._strong_owned: OrderedDict[int, object] = OrderedDict() + + async def resolve(self, request: RequestT) -> Workflow: + """Resolve one fresh built graph without sharing executors, agents, clients, or providers.""" + result = self.source(request) + workflow = await result if inspect.isawaitable(result) else result + if not isinstance(workflow, Workflow): + raise TypeError( + "The workflow factory must return a built Workflow, not an agent, WorkflowAgent, or WorkflowBuilder." + ) + resources: list[object] = [] + + def collect(graph: Workflow) -> None: + if ( + graph.status != WorkflowRunState.IDLE + or graph.get_last_checkpoint_id() is not None + or graph._runner.state.export_state() # pyright: ignore[reportPrivateUsage] + or graph._is_run_active() # pyright: ignore[reportPrivateUsage] + ): + raise RuntimeError("Native hosting requires a freshly built, unrun workflow.") + if graph._runner_context.has_checkpointing(): # pyright: ignore[reportPrivateUsage] + raise RuntimeError("Native hosting owns checkpoint storage; build the workflow without a store.") + resources.extend((graph, graph._runner_context)) # pyright: ignore[reportPrivateUsage] + for executor in graph.get_executors_list(): + resources.append(executor) + if isinstance(executor, WorkflowExecutor): + collect(executor.workflow) + + collect(workflow) + for agent_value in workflow_agents(workflow): + resources.append(agent_value) + if isinstance(agent_value, RawAgent): + agent = cast(RawAgent[Any], agent_value) + resources.append(agent.client) + resources.extend(agent.context_providers) + resources.extend(agent.mcp_tools) + tools = agent.default_options.get("tools", ()) + if isinstance(tools, Sequence): + resources.extend(tool for tool in cast(Sequence[object], tools) if not isinstance(tool, Mapping)) + unique_resources = {id(resource): resource for resource in resources} + for identifier, resource in unique_resources.items(): + previous = self._owned.get(id(resource)) + if (previous is not None and previous() is resource) or self._strong_owned.get(identifier) is resource: + raise RuntimeError( + "Native workflow requests cannot share workflows, executors, agents, clients, or providers. " + "Use a request-aware factory that creates fresh instances." + ) + for identifier, resource in unique_resources.items(): + try: + self._owned[identifier] = weakref.ref(resource, lambda ref, key=identifier: self._forget(key, ref)) + except TypeError: + # Some valid slotted protocol implementations omit __weakref__. + # Keep a bounded exact-identity fallback rather than rejecting them + # or retaining unbounded request resources for the host lifetime. + self._strong_owned[identifier] = resource + self._strong_owned.move_to_end(identifier) + while len(self._strong_owned) > _MAX_STRONG_RESOURCE_IDENTITIES: + self._strong_owned.popitem(last=False) + return workflow + + def _forget(self, key: int, reference: weakref.ReferenceType[Any]) -> None: + if self._owned.get(key) is reference: + del self._owned[key] + + +StoreT = TypeVar("StoreT") + + +def _encoded_checkpoint_hash(value: Any) -> str: + """Hash the exact JSON-compatible representation acknowledged by storage.""" + payload = json.dumps(value, sort_keys=True, allow_nan=False, separators=(",", ":")).encode("utf-8") + return hashlib.sha256(payload).hexdigest() + + +def _select_latest_checkpoint(checkpoints: list[WorkflowCheckpoint]) -> WorkflowCheckpoint: + """Return the latest checkpoint, resolving timestamp ties through the lineage chain. + + Mirrors the tie-breaker used by the core checkpoint storages. Checkpoints taken at the + same superstep boundary can share a timestamp. Among the checkpoints with the newest + timestamp, one that another of them names as its ``previous_checkpoint_id`` is + superseded, so the result does not depend on the order the store lists keys in. + If several unsuperseded candidates remain, the one with the greatest ``checkpoint_id`` + is returned, so the choice among them is stable too. + """ + newest_timestamp = max(datetime.fromisoformat(cp.timestamp) for cp in checkpoints) + candidates = [cp for cp in checkpoints if datetime.fromisoformat(cp.timestamp) == newest_timestamp] + superseded_ids = {cp.previous_checkpoint_id for cp in candidates} + leaves = [cp for cp in candidates if cp.checkpoint_id not in superseded_ids] + return max(leaves or candidates, key=lambda cp: cp.checkpoint_id) + + +def _store_scope(config: AgentConfig, platform_context: FoundryAgentRequestContext) -> FoundryRequestScope | None: + if not config.is_hosted: + return None + return FoundryRequestScope.from_context(config, platform_context) + + +def _store_name(root: str, scope: FoundryRequestScope | None, context_id: str | None = None) -> str: + if scope is None: + return f"{root}/{context_id}" if context_id is not None else root + name = f"{root}/v2/{scope.storage_key}" + if context_id is not None: + name += f"/{hashlib.sha256(context_id.encode('utf-8')).hexdigest()[:24]}" + return name + + +class StoreProvider(ABC, Generic[StoreT]): + """Provide store for a hosting environment.""" + + @abstractmethod + def get_store(self, *, config: AgentConfig, platform_context: FoundryAgentRequestContext) -> StoreT: + """Get store for a hosting environment. + + Args: + config: The resolved agent server configuration. + platform_context: The request-scoped platform context for the current request. + + Returns: + The store instance for the given hosting environment. + """ + + +class ContextScopedStoreProvider(ABC, Generic[StoreT]): + """Provide a context-scoped store for a hosting environment. + + Use this when state must be queried or managed as a collection belonging to + one context, rather than accessed only by an individual item ID. The context + can be a conversation ID or another identifier that groups related state. + """ + + @abstractmethod + def get_store( + self, + *, + config: AgentConfig, + context_id: str, + platform_context: FoundryAgentRequestContext, + ) -> StoreT: + """Get a context-scoped store for a hosting environment. + + Args: + config: The resolved agent server configuration. + context_id: A string that uniquely identifies the context for which the store is scoped. + platform_context: The request-scoped platform context for the current request. + + Returns: + The store instance for the given hosting environment and context ID. + """ + + +# region Checkpoint persistence + + +class FoundryCheckpointStore: + """Checkpoint store backed by the `FoundryStateStore`.""" + + DEFAULT_ROOT_SCOPE = "checkpoints" + + def __init__( + self, + context_id: str, + platform_context: FoundryAgentRequestContext, + *, + allowed_checkpoint_types: list[str] | None = None, + scope: FoundryRequestScope | None = None, + ) -> None: + """Initialize a Foundry-scoped checkpoint store for the given context ID. + + Args: + context_id: A string that uniquely identifies the context for which the checkpoint store is scoped. + This can be used to isolate checkpoints for different workflow runs. + platform_context: The request-scoped platform context for the current request. + allowed_checkpoint_types: Additional types (beyond the built-in safe set + and framework types) that are permitted during checkpoint + deserialization. Each entry should be a ``"module:qualname"`` + string (e.g., ``"my_app.models:MyState"``). + scope: Trusted request scope for hosted sandbox isolation. + """ + if not context_id: + raise ValueError("context_id must be provided to initialize a FoundryCheckpointStore.") + + self.context_id = context_id + self.platform_context = platform_context + self._allowed_types: frozenset[str] = frozenset(allowed_checkpoint_types or []) + self._scope = scope + + async def _get_store(self) -> FoundryStateStore: + return await FoundryStateStore.get_or_create( + _store_name(self.DEFAULT_ROOT_SCOPE, self._scope, self.context_id), + user_isolation=True, + ) + + async def save(self, checkpoint: WorkflowCheckpoint) -> CheckpointID: + """Save a workflow checkpoint to the store. + + Args: + checkpoint: The workflow checkpoint to save. + + The checkpoint will be serialized to a dictionary and then encoded before being stored. + + Returns: + The ID of the saved checkpoint. + """ + from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value + + encoded_checkpoint = encode_checkpoint_value(checkpoint.to_dict()) + + store = await self._get_store() + async with store: + await store.set_item(checkpoint.checkpoint_id, encoded_checkpoint, call_id=self.platform_context.call_id) + return checkpoint.checkpoint_id + + async def _load_encoded(self, checkpoint_id: CheckpointID) -> Any: + store = await self._get_store() + async with store: + item = await store.get_item(checkpoint_id, call_id=self.platform_context.call_id) + if item is None: + raise WorkflowCheckpointException(f"No checkpoint found with ID {checkpoint_id}") + return item.value + + async def load(self, checkpoint_id: CheckpointID) -> WorkflowCheckpoint: + """Load a workflow checkpoint from the store. + + Args: + checkpoint_id: The ID of the checkpoint to load. + + Returns: + The loaded workflow checkpoint. + + Raises: + WorkflowCheckpointException: If no checkpoint is found with the given ID. + """ + from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value + + encoded = await self._load_encoded(checkpoint_id) + return WorkflowCheckpoint.from_dict(decode_checkpoint_value(encoded, allowed_types=self._allowed_types)) + + async def load_with_hash(self, checkpoint_id: CheckpointID) -> tuple[WorkflowCheckpoint, str]: + """Load a checkpoint and hash its exact persisted encoding. + + Pickle is not a canonical serialization across Python versions or repeated + encodes. Hashing the acknowledged storage value preserves exact tamper + detection without rejecting an unchanged checkpoint after deserialization. + """ + from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value + + encoded = await self._load_encoded(checkpoint_id) + checkpoint = WorkflowCheckpoint.from_dict(decode_checkpoint_value(encoded, allowed_types=self._allowed_types)) + return checkpoint, _encoded_checkpoint_hash(encoded) + + async def list_checkpoints(self, *, workflow_name: str) -> list[WorkflowCheckpoint]: + """List all workflow checkpoints for a given workflow name.""" + from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value + + store = await self._get_store() + checkpoints: list[WorkflowCheckpoint] = [] + after: str | None = None + async with store: + while True: + page = await store.list_keys(after=after, call_id=self.platform_context.call_id) + for item_key in page.keys: + item = await store.get_item(item_key.key, call_id=self.platform_context.call_id) + if item is None: + continue + checkpoint = WorkflowCheckpoint.from_dict( + decode_checkpoint_value(item.value, allowed_types=self._allowed_types) + ) + if checkpoint.workflow_name == workflow_name: + checkpoints.append(checkpoint) + if not page.has_more or page.last_id is None: + break + after = page.last_id + return checkpoints + + async def delete(self, checkpoint_id: CheckpointID) -> bool: + store = await self._get_store() + async with store: + deleted_item = await store.delete_item(checkpoint_id, call_id=self.platform_context.call_id) + return deleted_item.id is not None + + async def get_latest(self, *, workflow_name: str) -> WorkflowCheckpoint | None: + checkpoints = await self.list_checkpoints(workflow_name=workflow_name) + if not checkpoints: + return None + return _select_latest_checkpoint(checkpoints) + + async def list_checkpoint_ids(self, *, workflow_name: str) -> list[CheckpointID]: + checkpoints = await self.list_checkpoints(workflow_name=workflow_name) + return [checkpoint.checkpoint_id for checkpoint in checkpoints] + + +class CheckpointStoreProvider(ContextScopedStoreProvider[CheckpointStorage]): + """Provide workflow checkpoint store scoped to a context. + + A workflow context can contain multiple checkpoints. Scoping establishes the + collection boundary used to list checkpoints, restore the latest checkpoint, + and clean up older checkpoints without affecting another workflow context. + + This defaults to using the `FoundryCheckpointStore` in all environments. + """ + + def __init__(self, *, allowed_checkpoint_types: list[str] | None = None) -> None: + """Initialize the provider. + + Args: + allowed_checkpoint_types: Additional types (beyond the built-in safe set + and framework types) that are permitted during checkpoint + deserialization, forwarded to every store this provider creates. + Each entry should be a ``"module:qualname"`` string + (e.g., ``"my_app.models:MyState"``). + """ + self._allowed_checkpoint_types = allowed_checkpoint_types + + def validate_checkpoint_value(self, value: object) -> None: + """Verify a value can round-trip through this provider's restricted decoder. + + Native hosts use this before claiming a stored typed workflow turn so an + omitted application allowlist fails before executors or tools run. + """ + from agent_framework._workflows._checkpoint_encoding import ( + decode_checkpoint_value, + encode_checkpoint_value, + ) + + decode_checkpoint_value( + encode_checkpoint_value(value), + allowed_types=frozenset(self._allowed_checkpoint_types or ()), + ) + + def get_store( + self, + *, + config: AgentConfig, + context_id: str, + platform_context: FoundryAgentRequestContext, + ) -> CheckpointStorage: + """Get checkpoint store for the requested hosting environment.""" + if not context_id: + raise ValueError("context_id must be provided to get a checkpoint store.") + + return self.get_store_for_scope( + context_id=context_id, + platform_context=platform_context, + scope=_store_scope(config, platform_context), + ) + + def get_store_for_scope( + self, + *, + scope: FoundryRequestScope | None, + context_id: str, + platform_context: FoundryAgentRequestContext, + ) -> CheckpointStorage: + """Create a checkpoint store using a host's already validated scope. + + Native hosts use this after protocol-specific identity resolution. + ``get_store`` retains its strict configuration-based scope validation. + + Args: + scope: Trusted user and sandbox scope resolved by the protocol host. + context_id: Native workflow lineage's checkpoint collection. + platform_context: The current request's storage authorization context. + """ + return FoundryCheckpointStore( + context_id, + platform_context, + allowed_checkpoint_types=self._allowed_checkpoint_types, + scope=scope, + ) + + +# endregion Checkpoint persistence + +# region Agent session persistence + + +class FoundryAgentSessionStore(SessionStore): + """Agent session store backed by the `FoundryStateStore`.""" + + DEFAULT_ROOT_SCOPE = "agent_sessions" + + def __init__( + self, + platform_context: FoundryAgentRequestContext, + *, + store_name: str | None = None, + scope: FoundryRequestScope | None = None, + ) -> None: + """Initialize session storage. + + Args: + platform_context: The request-scoped platform context. + store_name: Logical state-store name. Defaults to `agent_sessions`. + scope: Trusted Foundry request scope for sandbox-specific storage. + """ + if store_name is not None and (not isinstance(store_name, str) or not store_name.strip()): + raise ValueError("store_name must be a non-empty string") + self.platform_context = platform_context + self._store_name = self.DEFAULT_ROOT_SCOPE if store_name is None else store_name + self._scope = scope + self._etags: dict[str, str | None] = {} + + async def _get_store(self) -> FoundryStateStore: + return await FoundryStateStore.get_or_create( + _store_name(self._store_name, self._scope), + user_isolation=True, + ) + + async def get(self, session_id: str) -> AgentSession | None: + store = await self._get_store() + async with store: + item = await store.get_item(session_id, call_id=self.platform_context.call_id) + if item is None: + self._etags[session_id] = None + return None + if not item.etag: + raise RuntimeError("Foundry agent session storage returned a loaded session without an ETag.") + self._etags[session_id] = item.etag + return AgentSession.from_dict(item.value) + + async def set(self, session_id: str, session: AgentSession) -> None: + store = await self._get_store() + async with store: + try: + etag = self._etags.get(session_id) + if etag is not None: + result = await store.set_item( + session_id, session.to_dict(), if_match=etag, call_id=self.platform_context.call_id + ) + elif self._scope is not None or session_id in self._etags: + result = await store.create_item( + session_id, session.to_dict(), call_id=self.platform_context.call_id + ) + else: + result = await store.set_item(session_id, session.to_dict(), call_id=self.platform_context.call_id) + except (FoundryStorageConflictError, FoundryStoragePreconditionError) as exc: + raise RuntimeError("Another request advanced this agent session; reload before writing.") from exc + if not result.etag: + raise RuntimeError("Foundry agent session storage did not return an ETag after saving.") + self._etags[session_id] = result.etag + + async def delete(self, session_id: str) -> None: + store = await self._get_store() + async with store: + await store.delete_item(session_id, call_id=self.platform_context.call_id) + self._etags.pop(session_id, None) + + +class AgentSessionStoreProvider(StoreProvider[SessionStore]): + """Provide agent session store for the active hosting environment. + + This defaults to using the `FoundryAgentSessionStore` in all environments. + """ + + def __init__(self, *, store_name: str | None = None) -> None: + """Initialize the provider. + + Args: + store_name: Logical state-store name. Defaults to `agent_sessions`. + """ + if store_name is not None and (not isinstance(store_name, str) or not store_name.strip()): + raise ValueError("store_name must be a non-empty string") + self._store_name = store_name + + def _get_scoped_store( + self, platform_context: FoundryAgentRequestContext, scope: FoundryRequestScope | None + ) -> SessionStore: + return FoundryAgentSessionStore( + platform_context, + store_name=self._store_name, + scope=scope, + ) + + def get_store(self, *, config: AgentConfig, platform_context: FoundryAgentRequestContext) -> SessionStore: + """Get agent session store for the requested hosting environment.""" + return self._get_scoped_store(platform_context, _store_scope(config, platform_context)) + + +# endregion Agent session persistence + + +# region Native workflow state + +_CONFLICT = "Another request advanced this workflow. Reload its current response or start a fresh lineage." +_BLOCKED = "This workflow turn was interrupted or failed; start a fresh workflow lineage to avoid replaying effects." +logger = logging.getLogger(__name__) + + +class WorkflowConflictError(RuntimeError): + """A scoped workflow head, graph, or reply authority no longer permits this turn.""" + + +class WorkflowBlockedError(WorkflowConflictError): + """A failed or cancelled turn blocks its lineage to prevent uncertain effect replay.""" + + +def _key(kind: str, identifier: str) -> str: + if not isinstance(identifier, str) or not identifier: + raise ValueError("A non-empty workflow state identifier is required.") + return f"{kind}-{hashlib.sha256(identifier.encode('utf-8')).hexdigest()}" + + +def _json_object(value: Mapping[str, Any]) -> dict[str, Any]: + """Validate snapshots without arbitrary encoders or success-shaped string fallbacks.""" + if not isinstance(value, Mapping) or any(not isinstance(key, str) for key in value): + raise TypeError("A workflow output snapshot must be a JSON object.") + return cast(dict[str, Any], json.loads(json.dumps(dict(value), allow_nan=False))) + + +def _checkpoint_hash(checkpoint: WorkflowCheckpoint) -> str: + encoded = encode_checkpoint_value(checkpoint.to_dict()) + return hashlib.sha256(json.dumps(encoded, sort_keys=True, allow_nan=False).encode("utf-8")).hexdigest() + + +@dataclass(frozen=True) +class WorkflowBinding: + """The exact checkpoint for one outer response, not a sandbox or provider session.""" + + response_id: str + checkpoint_id: str | None + lineage_id: str + workflow_name: str + graph_hash: str + scope_key: str + checkpoint_hash: str | None = None + conversation_id: str | None = None + previous_response_id: str | None = None + + @classmethod + def from_value(cls, value: Any, scope: FoundryRequestScope) -> WorkflowBinding: + if not isinstance(value, dict): + raise ValueError("Invalid persisted workflow binding.") + value = cast(dict[str, Any], value) + if set(value) != set(cls.__dataclass_fields__): + raise ValueError("Invalid persisted workflow binding.") + for name in ("response_id", "lineage_id", "workflow_name", "graph_hash", "scope_key"): + if not isinstance(value[name], str) or not value[name]: + raise ValueError("Invalid persisted workflow binding.") + for name in ("checkpoint_id", "checkpoint_hash", "conversation_id", "previous_response_id"): + if value[name] is not None and (not isinstance(value[name], str) or not value[name]): + raise ValueError("Invalid persisted workflow binding.") + if (value["checkpoint_id"] is None) != (value["checkpoint_hash"] is None): + raise ValueError("Invalid persisted workflow checkpoint identity.") + binding = cls(**value) + if binding.scope_key != scope.storage_key: + raise PermissionError("The workflow state is not available in this trusted scope.") + return binding + + +@dataclass(frozen=True) +class WorkflowRecord: + """A per-response checkpoint/output pair retained independently of a later head.""" + + binding: WorkflowBinding + owner: str + status: Literal["active", "completed", "blocked"] + snapshot: dict[str, Any] | None = None + approvals: dict[str, str] | None = None + + @classmethod + def from_value(cls, value: Any, scope: FoundryRequestScope) -> WorkflowRecord: + if not isinstance(value, dict): + raise ValueError("Invalid persisted workflow response pair.") + value = cast(dict[str, Any], value) + if set(value) != set(cls.__dataclass_fields__): + raise ValueError("Invalid persisted workflow response pair.") + binding = WorkflowBinding.from_value(value["binding"], scope) + if ( + not isinstance(value["owner"], str) + or not value["owner"] + or value["status"] + not in ( + "active", + "completed", + "blocked", + ) + ): + raise ValueError("Invalid persisted workflow response pair.") + snapshot = value["snapshot"] + if snapshot is not None: + snapshot = _json_object(snapshot) + raw_approvals = value["approvals"] + approvals: dict[str, str] | None = None + if raw_approvals is not None: + if not isinstance(raw_approvals, dict): + raise ValueError("Invalid persisted workflow reply authority.") + approvals = {} + for wire_id, request_id in cast(dict[object, object], raw_approvals).items(): + if not isinstance(wire_id, str) or not wire_id or not isinstance(request_id, str) or not request_id: + raise ValueError("Invalid persisted workflow reply authority.") + approvals[wire_id] = request_id + if value["status"] == "completed" and (binding.checkpoint_id is None or snapshot is None): + raise ValueError("A completed workflow response requires an exact checkpoint/output pair.") + return cls(binding, value["owner"], value["status"], snapshot, approvals) + + +@dataclass(frozen=True) +class WorkflowHead: + """One CAS authority for a lineage, including its optional named conversation.""" + + lineage_id: str + conversation_id: str | None + binding: WorkflowBinding | None = None + response_id: str | None = None + owner: str | None = None + blocked: bool = False + + @classmethod + def from_value(cls, value: Any, scope: FoundryRequestScope) -> WorkflowHead: + if not isinstance(value, dict): + raise ValueError("Invalid persisted workflow head.") + value = cast(dict[str, Any], value) + if set(value) != set(cls.__dataclass_fields__): + raise ValueError("Invalid persisted workflow head.") + if not isinstance(value["lineage_id"], str) or not value["lineage_id"] or type(value["blocked"]) is not bool: + raise ValueError("Invalid persisted workflow head.") + for name in ("conversation_id", "response_id", "owner"): + if value[name] is not None and (not isinstance(value[name], str) or not value[name]): + raise ValueError("Invalid persisted workflow head.") + if (value["owner"] is None) != (value["response_id"] is None): + raise ValueError("Invalid persisted workflow claim.") + binding = WorkflowBinding.from_value(value["binding"], scope) if value["binding"] is not None else None + if binding is not None and ( + binding.lineage_id != value["lineage_id"] or binding.conversation_id != value["conversation_id"] + ): + raise ValueError("Invalid persisted workflow head binding.") + return cls( + value["lineage_id"], + value["conversation_id"], + binding, + value["response_id"], + value["owner"], + value["blocked"], + ) + + +class FoundryWorkflowBindingStore: + """Persist scoped per-response pairs and fenced linear workflow claims.""" + + def __init__(self, scope: FoundryRequestScope) -> None: + self.scope = scope + + async def _get_store(self) -> FoundryStateStore: + return await FoundryStateStore.get_or_create( + f"workflow_bindings/v2/{self.scope.storage_key}", user_isolation=True + ) + + async def get_response(self, response_id: str) -> tuple[WorkflowRecord | None, str | None]: + store = await self._get_store() + async with store: + item = await store.get_item(_key("response", response_id), call_id=self.scope.call_id) + if item is None: + return None, None + record = WorkflowRecord.from_value(item.value, self.scope) + if record.binding.response_id != response_id or not item.etag: + raise ValueError("Invalid persisted workflow response identity or ETag.") + return record, item.etag + + async def get_head(self, lineage_id: str, conversation_id: str | None) -> tuple[WorkflowHead | None, str | None]: + store = await self._get_store() + key = _key("conversation", conversation_id) if conversation_id is not None else _key("lineage", lineage_id) + async with store: + item = await store.get_item(key, call_id=self.scope.call_id) + if item is None: + return None, None + head = WorkflowHead.from_value(item.value, self.scope) + if ( + head.conversation_id != conversation_id + or (conversation_id is None and head.lineage_id != lineage_id) + or not item.etag + ): + raise ValueError("Invalid persisted workflow head identity or ETag.") + return head, item.etag + + async def save_response(self, record: WorkflowRecord, *, expected_etag: str | None) -> str: + if record.binding.scope_key != self.scope.storage_key: + raise PermissionError("Cannot write workflow state outside its trusted scope.") + return await self._write(_key("response", record.binding.response_id), asdict(record), expected_etag) + + async def save_head(self, head: WorkflowHead, *, expected_etag: str | None) -> str: + if head.binding is not None and head.binding.scope_key != self.scope.storage_key: + raise PermissionError("Cannot write workflow state outside its trusted scope.") + key = ( + _key("conversation", head.conversation_id) + if head.conversation_id is not None + else _key("lineage", head.lineage_id) + ) + return await self._write(key, asdict(head), expected_etag) + + async def delete_response(self, response_id: str, *, expected_etag: str) -> bool: + """Conditionally delete a superseded response record.""" + store = await self._get_store() + async with store: + try: + deleted = await store.delete_item( + _key("response", response_id), + if_match=expected_etag, + call_id=self.scope.call_id, + ) + except (FoundryStorageConflictError, FoundryStoragePreconditionError) as exc: + raise WorkflowConflictError(_CONFLICT) from exc + return deleted.id is not None + + async def _write(self, key: str, value: dict[str, Any], expected_etag: str | None) -> str: + store = await self._get_store() + async with store: + try: + if expected_etag is None: + result = await store.create_item(key, value, call_id=self.scope.call_id) + else: + result = await store.set_item(key, value, if_match=expected_etag, call_id=self.scope.call_id) + except (FoundryStorageConflictError, FoundryStoragePreconditionError) as exc: + raise WorkflowConflictError(_CONFLICT) from exc + if not result.etag: + raise RuntimeError("Workflow storage did not acknowledge a conditional write with an ETag.") + return result.etag + + +class _CheckpointWrites: + """Track acknowledged core saves, never selecting an unrelated latest checkpoint.""" + + def __init__(self, storage: CheckpointStorage, run: HostedWorkflowRun) -> None: + self.storage = storage + self.run = run + self.last_checkpoint: WorkflowCheckpoint | None = None + self.acknowledged_ids: set[str] = set() + + async def save(self, checkpoint: WorkflowCheckpoint) -> str: + await self.run.assert_claim() + validate_workflow_provider_state(self.run.workflow, checkpoint) + identifier = await self.storage.save(checkpoint) + if identifier != checkpoint.checkpoint_id: + raise RuntimeError("Workflow checkpoint storage acknowledged a different checkpoint.") + self.last_checkpoint = checkpoint + self.acknowledged_ids.add(identifier) + return identifier + + async def load(self, checkpoint_id: str) -> WorkflowCheckpoint: + checkpoint = await self.storage.load(checkpoint_id) + # Core checkpoint restoration follows run-kwargs setup. Retain fresh request + # kwargs rather than restoring the previous request's private call context. + fresh = self.run.workflow._runner.state.export_state() # pyright: ignore[reportPrivateUsage] + state = dict(checkpoint.state) + for key in (WORKFLOW_RUN_KWARGS_KEY, RESOLVED_WORKFLOW_RUN_KWARGS_KEY, ROUTED_WORKFLOW_RUN_KWARGS_KEY): + state[key] = fresh.get(key, {}) + return replace(checkpoint, state=state) + + async def load_persisted(self, checkpoint_id: str) -> tuple[WorkflowCheckpoint, str]: + """Load a checkpoint with a process-stable integrity hash when supported.""" + load_with_hash = getattr(cast(Any, self.storage), "load_with_hash", None) + if callable(load_with_hash): + loader = cast( + Callable[[str], Awaitable[tuple[WorkflowCheckpoint, str]]], + load_with_hash, + ) + checkpoint, checkpoint_hash = await loader(checkpoint_id) + if not isinstance(checkpoint, WorkflowCheckpoint) or not isinstance(checkpoint_hash, str): + raise TypeError("Checkpoint storage returned an invalid checkpoint integrity pair.") + return checkpoint, checkpoint_hash + checkpoint = await self.storage.load(checkpoint_id) + return checkpoint, _checkpoint_hash(checkpoint) + + async def list_checkpoints(self, *, workflow_name: str) -> list[WorkflowCheckpoint]: + return await self.storage.list_checkpoints(workflow_name=workflow_name) + + async def list_checkpoint_ids(self, *, workflow_name: str) -> list[str]: + return await self.storage.list_checkpoint_ids(workflow_name=workflow_name) + + async def get_latest(self, *, workflow_name: str) -> WorkflowCheckpoint | None: + return await self.storage.get_latest(workflow_name=workflow_name) + + async def delete(self, checkpoint_id: str) -> bool: + return await self.storage.delete(checkpoint_id) + + +class HostedWorkflowRun: + """A native turn with explicit validation, claim, pairing, commit, and abort boundaries. + + Protocol hosts own output encoding and lifecycle signals. Closing ``events`` + never commits a turn. A failed or cancelled claim is blocked rather than + releasing potentially consumed approvals or replaying external side effects. + """ + + def __init__( + self, + workflow: Workflow, + scope: FoundryRequestScope, + *, + stored: bool, + recovery: bool, + fresh_factory: bool, + reclaim_superseded: bool, + ) -> None: + self.workflow = workflow + self.scope = scope + self.stored = stored + self.recovery = recovery + self.fresh_factory = fresh_factory + self.reclaim_superseded = reclaim_superseded + self.store = FoundryWorkflowBindingStore(scope) + self.binding: WorkflowBinding + self._head: WorkflowHead | None = None + self._head_etag: str | None = None + self._record: WorkflowRecord | None = None + self._record_etag: str | None = None + self._storage: _CheckpointWrites | None = None + self._checkpoint: WorkflowCheckpoint | None = None + self._reply_approvals: dict[str, str] = {} + self._turn: WorkflowTurn[Any] | None = None + self._client_kwargs = WorkflowInvocationKwargs() + self._function_kwargs = WorkflowInvocationKwargs() + self._claimed = False + self._finalized = False + self.already_completed = False + + @classmethod + async def prepare( + cls, + workflow: Workflow, + *, + scope: FoundryRequestScope, + response_id: str, + config: AgentConfig, + platform_context: FoundryAgentRequestContext, + checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage], + previous_response_id: str | None = None, + conversation_id: str | None = None, + lineage_id: str | None = None, + stored: bool = True, + recovery: bool = False, + replay_completed: bool = False, + fresh_factory: bool = False, + reclaim_superseded: bool = False, + ) -> HostedWorkflowRun: + """Read exact continuation state without claiming, consuming replies, or executing.""" + if not workflow.name or not workflow.graph_signature_hash: + raise ValueError("Native workflows require a stable, non-empty workflow name and graph identity.") + if previous_response_id is not None and conversation_id is not None: + raise ValueError("A workflow turn cannot combine previous_response_id and conversation.") + if not stored and (recovery or previous_response_id is not None or conversation_id is not None): + raise ValueError("store=false workflow requests cannot continue stored state; start a one-shot turn.") + run = cls( + workflow, + scope, + stored=stored, + recovery=recovery, + fresh_factory=fresh_factory, + reclaim_superseded=reclaim_superseded, + ) + run.binding = WorkflowBinding( + response_id=response_id, + checkpoint_id=None, + lineage_id=lineage_id or response_id, + workflow_name=workflow.name, + graph_hash=workflow.graph_signature_hash, + scope_key=scope.storage_key, + conversation_id=conversation_id, + previous_response_id=previous_response_id, + ) + if not stored: + return run + + own, own_etag = await run.store.get_response(response_id) + if not recovery and replay_completed and own is not None and own.status == "completed": + recovery = True + run.recovery = True + if recovery and own is not None: + run._record, run._record_etag = own, own_etag + run.binding = own.binding + run._reply_approvals = own.approvals or {} + elif own is not None: + raise WorkflowConflictError( + "This workflow response already exists; retrieve it instead of executing it again." + ) + + previous: WorkflowBinding | None = None + if previous_response_id is not None and (not recovery or own is None): + record, _ = await run.store.get_response(previous_response_id) + if record is None or record.status != "completed": + raise ValueError("The previous response has no completed workflow checkpoint in this trusted scope.") + previous = record.binding + if not recovery: + run._reply_approvals = record.approvals or {} + if previous.conversation_id is not None: + raise ValueError("A named workflow conversation cannot be forked through previous_response_id.") + run.binding = replace(run.binding, lineage_id=previous.lineage_id) + + head, etag = await run.store.get_head(run.binding.lineage_id, run.binding.conversation_id) + run._head, run._head_etag = head, etag + if recovery: + if head is None: + raise RuntimeError("The workflow recovery authority is unavailable; start a fresh lineage.") + if own is None: + if head.response_id != response_id or head.owner is None or head.blocked: + raise WorkflowConflictError(_CONFLICT) + run.binding = replace(run.binding, lineage_id=head.lineage_id) + blocked_record = WorkflowRecord(run.binding, head.owner, "blocked") + await run.store.save_response(blocked_record, expected_etag=None) + await run.store.save_head(replace(head, blocked=True), expected_etag=etag) + raise WorkflowBlockedError(_BLOCKED) + if own.status == "blocked": + if head.response_id == response_id and head.owner == own.owner and not head.blocked: + await run.store.save_head(replace(head, blocked=True), expected_etag=etag) + raise WorkflowBlockedError(_BLOCKED) + if own.binding.checkpoint_id is None or own.snapshot is None: + if head.response_id != response_id or head.owner != own.owner or head.blocked: + raise WorkflowConflictError(_CONFLICT) + blocked_record = replace(own, status="blocked") + await run.store.save_response(blocked_record, expected_etag=own_etag) + await run.store.save_head(replace(head, blocked=True), expected_etag=etag) + raise WorkflowBlockedError(_BLOCKED) + if own.status == "completed": + current = head.binding + visited: set[str] = set() + while current is not None and current != own.binding: + if current.response_id in visited or current.previous_response_id is None: + current = None + break + visited.add(current.response_id) + parent, _ = await run.store.get_response(current.previous_response_id) + current = parent.binding if parent is not None and parent.status == "completed" else None + if current is not None: + run.already_completed = True + elif head.response_id != response_id or head.blocked: + raise WorkflowConflictError(_CONFLICT) + else: + run._finalized = True + elif head.response_id != response_id or head.owner != own.owner or head.blocked: + raise WorkflowConflictError(_CONFLICT) + previous = run.binding + elif head is not None: + if head.blocked: + raise WorkflowBlockedError(_BLOCKED) + if head.response_id is not None: + active_record, _ = await run.store.get_response(head.response_id) + if active_record is None and head.owner is not None: + interrupted_binding = WorkflowBinding( + response_id=head.response_id, + checkpoint_id=None, + lineage_id=head.lineage_id, + workflow_name=workflow.name, + graph_hash=workflow.graph_signature_hash, + scope_key=scope.storage_key, + conversation_id=head.conversation_id, + previous_response_id=head.binding.response_id if head.binding is not None else None, + ) + await run.store.save_response( + WorkflowRecord(interrupted_binding, head.owner, "blocked"), + expected_etag=None, + ) + await run.store.save_head(replace(head, blocked=True), expected_etag=etag) + raise WorkflowBlockedError(_BLOCKED) + raise WorkflowConflictError( + "This workflow has an in-flight turn. Wait for it or start a fresh lineage." + ) + if previous is not None and head.binding != previous: + raise WorkflowConflictError("This workflow continuation is stale or forked; use its current response.") + previous = head.binding + if previous is not None: + record, _ = await run.store.get_response(previous.response_id) + if record is None or record.status != "completed" or record.binding != previous: + raise ValueError("The workflow head has no exact completed response binding.") + run._reply_approvals = record.approvals or {} + run.binding = replace( + run.binding, + lineage_id=head.lineage_id, + previous_response_id=previous.response_id if previous is not None else None, + ) + elif previous is not None: + raise ValueError("The workflow lineage authority is unavailable in this trusted scope.") + + context_id = _key("native", f"{scope.storage_key}:{run.binding.lineage_id}") + storage = ( + checkpoint_store_provider.get_store_for_scope( + scope=scope, context_id=context_id, platform_context=platform_context + ) + if isinstance(checkpoint_store_provider, CheckpointStoreProvider) + else checkpoint_store_provider.get_store( + config=config, context_id=context_id, platform_context=platform_context + ) + ) + run._storage = _CheckpointWrites(storage, run) + if previous is not None: + if previous.workflow_name != workflow.name or previous.graph_hash != workflow.graph_signature_hash: + raise WorkflowConflictError( + "The stored workflow graph is incompatible. Rebuild its original graph or start a fresh lineage." + ) + if previous.checkpoint_id is None: + raise ValueError("The workflow response has no acknowledged checkpoint.") + checkpoint, checkpoint_hash = await run._storage.load_persisted(previous.checkpoint_id) + if ( + checkpoint.checkpoint_id != previous.checkpoint_id + or checkpoint.workflow_name != workflow.name + or checkpoint.graph_signature_hash != workflow.graph_signature_hash + or checkpoint_hash != previous.checkpoint_hash + ): + raise ValueError("The exact workflow checkpoint does not match its response binding.") + run._checkpoint = checkpoint + validate_workflow_provider_state(workflow, checkpoint) + return run + + @property + def snapshot(self) -> dict[str, Any] | None: + """The exact same-response recovery snapshot, independent of the current lineage head.""" + return self._record.snapshot if self._record is not None else None + + @property + def approvals(self) -> Mapping[str, str]: + """Caller-facing approval aliases bound to the loaded response's exact checkpoint.""" + return self._reply_approvals + + @property + def has_completed_output(self) -> bool: + """Whether recovery can deliver already finalized output without rerunning executors.""" + return self._record is not None and self._record.status == "completed" + + @property + def pending_requests(self) -> Mapping[str, WorkflowEvent[Any]]: + """Pending request authority from the exact acknowledged checkpoint.""" + checkpoint = self.current_checkpoint + return checkpoint.pending_request_info_events if checkpoint is not None else {} + + @property + def current_checkpoint(self) -> WorkflowCheckpoint | None: + """The last acknowledged checkpoint from this run, or its exact restoration checkpoint.""" + if self._storage is not None and self._storage.last_checkpoint is not None: + return self._storage.last_checkpoint + return self._checkpoint + + @property + def checkpoint_id(self) -> str | None: + """The exact acknowledged checkpoint ID, never a latest-checkpoint query.""" + checkpoint = self.current_checkpoint + return checkpoint.checkpoint_id if checkpoint is not None else None + + def recovery_turn(self) -> WorkflowTurn[Any]: + """Restore application kwargs without re-parsing already consumed approval input. + + The checkpoint, rather than a new start input or replayed reply, is the + recovery action. Host-injected identity and storage controls are replaced + by ``prepare_workflow_kwargs`` for the current trusted request. + """ + if not self.recovery or self._checkpoint is None: + raise RuntimeError("A recovery turn requires an exact restored checkpoint.") + resolved = self._checkpoint.state.get(RESOLVED_WORKFLOW_RUN_KWARGS_KEY, {}) + if not isinstance(resolved, Mapping): + raise ValueError("Invalid persisted workflow invocation kwargs.") + resolved = cast(Mapping[str, Any], resolved) + + def invocation_kwargs(name: str) -> WorkflowInvocationKwargs: + values = resolved.get(name, {}) + if not isinstance(values, Mapping): + raise ValueError("Invalid persisted workflow invocation kwargs.") + values = cast(Mapping[str, Any], values) + global_kwargs = dict(values.get("global_kwargs", {})) + executor_kwargs = {key: dict(value) for key, value in values.get("executor_kwargs", {}).items()} + if name == "client_kwargs": + for kwargs in (global_kwargs, *executor_kwargs.values()): + kwargs.pop("additional_function_arguments", None) + if "options" in kwargs: + kwargs["options"] = dict(kwargs["options"]) + kwargs["options"].pop("store", None) + return WorkflowInvocationKwargs(global_kwargs, executor_kwargs) + + return WorkflowTurn( + input=self._checkpoint, + client_kwargs=invocation_kwargs("client_kwargs"), + function_invocation_kwargs=invocation_kwargs("function_invocation_kwargs"), + ) + + def validate_turn(self, turn: WorkflowTurn[Any]) -> WorkflowTurn[Any]: + """Validate all input/replies and existing workflow kwargs before claiming authority.""" + if not isinstance(turn, WorkflowTurn): + raise TypeError("The workflow parser must return WorkflowTurn.") + self._client_kwargs, self._function_kwargs = prepare_workflow_kwargs( + self.workflow, + turn, + self.scope, + stored=self.stored, + fresh_factory=self.fresh_factory, + ) + if self.recovery: + self._turn = turn + return turn + pending = self.pending_requests + if turn.responses is not None: + if not self.stored or self._checkpoint is None: + raise ValueError("Workflow replies require a pending stored checkpoint.") + if set(turn.responses) != set(pending) or not pending: + raise ValueError("Workflow replies must answer exactly the complete pending request batch.") + replies: dict[str, Any] = {} + reply_ids: set[tuple[str, str]] = set() + for request_id, reply in turn.responses.items(): + event = pending[request_id] + try: + reply = _coerce_request_info_response(reply, event.response_type, request_id) + except (TypeError, ValueError) as exc: + raise ValueError("A workflow reply does not match its pending request type.") from exc + if isinstance(event.data, Content): + request = event.data + if isinstance(reply, Content): + identifier = reply.id if reply.type == "function_approval_response" else reply.call_id + if identifier is not None: + key = (reply.type, identifier) + if key in reply_ids: + raise ValueError("Duplicate workflow reply authority is not allowed.") + reply_ids.add(key) + if request.type == "function_approval_request" and ( + not isinstance(reply, Content) + or reply.type != "function_approval_response" + or reply.id != request.id + or type(reply.approved) is not bool + or reply.function_call is None + or request.function_call is None + or reply.function_call.to_dict() != request.function_call.to_dict() + ): + raise ValueError("The approval decision does not match its exact pending function call.") + if request.type == "function_call" and ( + not isinstance(reply, Content) + or reply.type != "function_result" + or reply.call_id != request.call_id + ): + raise ValueError("The function result does not match its pending call.") + if request.type == "computer_tool_call": + if not isinstance(reply, Content): + raise ValueError("A computer reply must be typed Content.") + _validate_computer_tool_result(request, reply) + replies[request_id] = reply + turn = replace(turn, responses=replies) + else: + if pending: + raise ValueError("This workflow is awaiting replies; do not replace them with new input.") + if self._checkpoint is not None and any(self._checkpoint.messages.values()): + raise ValueError("The workflow is still in flight; new input cannot replace its checkpoint.") + if not any(is_instance_of(turn.input, input_type) for input_type in self.workflow.input_types): + raise ValueError("The workflow parser's input does not match the start executor's type.") + self._turn = turn + return turn + + async def claim(self) -> None: + """Fence the exact validated turn before any executor or response handler runs.""" + if self._turn is None: + raise RuntimeError("Validate the workflow turn before claiming it.") + if self._claimed or self.already_completed: + return + if not self.stored: + self._claimed = True + return + owner = secrets.token_hex(16) + head = replace( + self._head or WorkflowHead(self.binding.lineage_id, self.binding.conversation_id), + response_id=self.binding.response_id, + owner=owner, + ) + self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) + self._head = head + self._claimed = True + record = ( + replace(self._record, owner=owner) + if self.recovery and self._record is not None + else WorkflowRecord(self.binding, owner, "active") + ) + self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) + self._record = record + await self._reclaim_completed_ancestors() + + async def assert_claim(self) -> None: + if not self._claimed: + raise RuntimeError("The workflow turn has not been claimed.") + if not self.stored: + return + head, etag = await self.store.get_head(self.binding.lineage_id, self.binding.conversation_id) + if head != self._head or etag != self._head_etag or head is None or head.blocked: + raise WorkflowConflictError(_CONFLICT) + + async def events( + self, + turn: WorkflowTurn[Any], + *, + client_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None, + function_invocation_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None, + ) -> AsyncGenerator[WorkflowEvent[Any]]: + """Execute core events, with explicit caller-owned commit and cancellation boundaries.""" + if self.already_completed or self.has_completed_output: + return + await self.assert_claim() + if self._turn is None: + raise RuntimeError("The workflow turn has not been validated.") + turn = self._turn + if self._checkpoint is not None and not self.recovery: + await self.workflow._runner.restore_checkpoint(self._checkpoint) # pyright: ignore[reportPrivateUsage] + stream = self.workflow.run( + None if self.recovery else turn.input, + responses=None if self.recovery else turn.responses, + checkpoint_id=self._checkpoint.checkpoint_id if self.recovery and self._checkpoint is not None else None, + checkpoint_storage=self._storage, + stream=True, + client_kwargs=client_kwargs if client_kwargs is not None else self._client_kwargs, + function_invocation_kwargs=( + function_invocation_kwargs if function_invocation_kwargs is not None else self._function_kwargs + ), + ) + try: + async for event in stream: + data = event.data + if ( + isinstance(data, (AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate)) + and data.continuation_token is not None + ): + raise RuntimeError("Native workflows cannot checkpoint unfinished provider background output.") + if event.type == "superstep_completed": + validate_workflow_provider_state(self.workflow) + if event.type == "request_info" and not self.stored: + raise ValueError("Approval and user-input continuation requires store=true.") + yield event + result = await stream.get_final_response() + if result.get_final_state() not in (WorkflowRunState.IDLE, WorkflowRunState.IDLE_WITH_PENDING_REQUESTS): + raise RuntimeError("The workflow did not finalize successfully.") + validate_workflow_provider_state(self.workflow) + self._finalized = True + finally: + await stream.close() + + async def stage( + self, + snapshot: Mapping[str, Any], + *, + approvals: Mapping[str, str] | None = None, + checkpoint_id: str | None = None, + ) -> None: + """CAS-pair output with an acknowledged core checkpoint before publishing its events.""" + if not self.stored: + return + snapshot_value = _json_object(snapshot) + checkpoint, checkpoint_hash = await self._get_checkpoint_with_hash(checkpoint_id or self.checkpoint_id) + if checkpoint is None: + raise RuntimeError("Output cannot be paired before a core checkpoint is acknowledged.") + if approvals is not None and any( + request_id not in checkpoint.pending_request_info_events for request_id in approvals.values() + ): + raise ValueError("Approval authority must belong to this exact pending checkpoint.") + await self.assert_claim() + if self._record is None: + raise RuntimeError("The workflow response pair has not been claimed.") + binding = replace(self.binding, checkpoint_id=checkpoint.checkpoint_id, checkpoint_hash=checkpoint_hash) + pending_approvals = { + wire_id: request_id + for wire_id, request_id in (approvals if approvals is not None else self._record.approvals or {}).items() + if request_id in checkpoint.pending_request_info_events + } + record = replace( + self._record, + binding=binding, + snapshot=snapshot_value, + approvals=pending_approvals, + ) + self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) + self.binding, self._record = binding, record + await self.assert_claim() + + async def get_checkpoint(self, checkpoint_id: str | None) -> WorkflowCheckpoint | None: + """Read a checkpoint acknowledged in this run, including a producer-stamped stream boundary.""" + checkpoint, _ = await self._get_checkpoint_with_hash(checkpoint_id) + return checkpoint + + async def _get_checkpoint_with_hash( + self, checkpoint_id: str | None + ) -> tuple[WorkflowCheckpoint | None, str | None]: + if checkpoint_id is None: + return None, None + if self._storage is None or ( + checkpoint_id not in self._storage.acknowledged_ids + and (self._checkpoint is None or checkpoint_id != self._checkpoint.checkpoint_id) + ): + raise ValueError("Output pairing requires a checkpoint acknowledged by this exact workflow turn.") + checkpoint, checkpoint_hash = await self._storage.load_persisted(checkpoint_id) + if ( + checkpoint.checkpoint_id != checkpoint_id + or checkpoint.workflow_name != self.workflow.name + or checkpoint.graph_signature_hash != self.workflow.graph_signature_hash + ): + raise ValueError("The acknowledged workflow checkpoint identity changed.") + validate_workflow_provider_state(self.workflow, checkpoint) + return checkpoint, checkpoint_hash + + async def commit(self, snapshot: Mapping[str, Any] | None = None) -> None: + """Commit only fully finalized, encoded output, before protocol success or done.""" + if self.already_completed: + return + if not self._finalized: + raise RuntimeError("A workflow cannot commit before successful core finalization.") + if not self.stored: + return + if snapshot is not None: + await self.stage(snapshot) + await self.assert_claim() + if self._record is None or self._record.snapshot is None or self.binding.checkpoint_id != self.checkpoint_id: + raise RuntimeError("The final workflow output has not been paired with its exact checkpoint.") + record = replace(self._record, status="completed") + self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) + self._record = record + head = WorkflowHead(self.binding.lineage_id, self.binding.conversation_id, self.binding) + self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) + self._head = head + self._claimed = False + + async def _reclaim_completed_ancestors(self) -> None: + """Best-effort reclaim after the new head is durably committed. + + Cleanup is oldest-first. If a deletion fails, newer records retain the + chain needed for a later successful commit to retry cleanup. + """ + if not self.reclaim_superseded or self._storage is None: + return + response_id = self.binding.previous_response_id + records: list[tuple[WorkflowRecord, str]] = [] + visited: set[str] = set() + try: + current_checkpoint_id = self.binding.checkpoint_id + checkpoint_ids = await self._storage.storage.list_checkpoint_ids(workflow_name=self.binding.workflow_name) + for checkpoint_id in checkpoint_ids: + if current_checkpoint_id is None or checkpoint_id != current_checkpoint_id: + await self._storage.storage.delete(checkpoint_id) + while response_id is not None: + if response_id in visited: + raise ValueError("Invalid cycle in superseded workflow response lineage.") + visited.add(response_id) + record, etag = await self.store.get_response(response_id) + if record is None or etag is None: + break + if ( + record.status != "completed" + or record.binding.scope_key != self.scope.storage_key + or record.binding.lineage_id != self.binding.lineage_id + or record.binding.conversation_id != self.binding.conversation_id + ): + break + records.append((record, etag)) + response_id = record.binding.previous_response_id + for record, etag in reversed(records): + await self.store.delete_response(record.binding.response_id, expected_etag=etag) + except Exception: + logger.exception("Failed to reclaim superseded native workflow state; a later commit can retry cleanup") + + async def abort(self) -> None: + """Block interrupted/failed authority instead of permitting unsafe reply or effect replay.""" + if not self.stored or not self._claimed or self.already_completed: + return + await self.assert_claim() + if self._record is not None: + record = replace(self._record, status="blocked") + self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) + self._record = record + if self._head is not None: + head = replace(self._head, blocked=True) + self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) + self._head = head + self._claimed = False + + +# endregion Native workflow state diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py deleted file mode 100644 index 539287adc74..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py +++ /dev/null @@ -1,514 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - - -import hashlib -import json -from abc import ABC, abstractmethod -from datetime import datetime -from typing import Any, Generic, Protocol, TypeVar - -from agent_framework import ( - AgentSession, - CheckpointID, - CheckpointStorage, - Content, - SessionStore, - WorkflowCheckpoint, - WorkflowCheckpointException, -) -from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext -from azure.ai.agentserver.core.storage import ( - FoundryStateStore, - FoundryStorageConflictError, - FoundryStoragePreconditionError, -) - -from ._scope import FoundryRequestScope - -StoreT = TypeVar("StoreT") - - -def _encoded_checkpoint_hash(value: Any) -> str: - """Hash the exact JSON-compatible representation acknowledged by storage.""" - payload = json.dumps(value, sort_keys=True, allow_nan=False, separators=(",", ":")).encode("utf-8") - return hashlib.sha256(payload).hexdigest() - - -def _select_latest_checkpoint(checkpoints: list[WorkflowCheckpoint]) -> WorkflowCheckpoint: - """Return the latest checkpoint, resolving timestamp ties through the lineage chain. - - Mirrors the tie-breaker used by the core checkpoint storages. Checkpoints taken at the - same superstep boundary can share a timestamp. Among the checkpoints with the newest - timestamp, one that another of them names as its ``previous_checkpoint_id`` is - superseded, so the result does not depend on the order the store lists keys in. - If several unsuperseded candidates remain, the one with the greatest ``checkpoint_id`` - is returned, so the choice among them is stable too. - """ - newest_timestamp = max(datetime.fromisoformat(cp.timestamp) for cp in checkpoints) - candidates = [cp for cp in checkpoints if datetime.fromisoformat(cp.timestamp) == newest_timestamp] - superseded_ids = {cp.previous_checkpoint_id for cp in candidates} - leaves = [cp for cp in candidates if cp.checkpoint_id not in superseded_ids] - return max(leaves or candidates, key=lambda cp: cp.checkpoint_id) - - -def _store_scope(config: AgentConfig, platform_context: FoundryAgentRequestContext) -> FoundryRequestScope | None: - if not config.is_hosted: - return None - return FoundryRequestScope.from_context(config, platform_context) - - -def _store_name(root: str, scope: FoundryRequestScope | None, context_id: str | None = None) -> str: - if scope is None: - return f"{root}/{context_id}" if context_id is not None else root - name = f"{root}/v2/{scope.storage_key}" - if context_id is not None: - name += f"/{hashlib.sha256(context_id.encode('utf-8')).hexdigest()[:24]}" - return name - - -class StoreProvider(ABC, Generic[StoreT]): - """Provide store for a hosting environment.""" - - @abstractmethod - def get_store(self, *, config: AgentConfig, platform_context: FoundryAgentRequestContext) -> StoreT: - """Get store for a hosting environment. - - Args: - config: The resolved agent server configuration. - platform_context: The request-scoped platform context for the current request. - - Returns: - The store instance for the given hosting environment. - """ - - -class ContextScopedStoreProvider(ABC, Generic[StoreT]): - """Provide a context-scoped store for a hosting environment. - - Use this when state must be queried or managed as a collection belonging to - one context, rather than accessed only by an individual item ID. The context - can be a conversation ID or another identifier that groups related state. - """ - - @abstractmethod - def get_store( - self, - *, - config: AgentConfig, - context_id: str, - platform_context: FoundryAgentRequestContext, - ) -> StoreT: - """Get a context-scoped store for a hosting environment. - - Args: - config: The resolved agent server configuration. - context_id: A string that uniquely identifies the context for which the store is scoped. - platform_context: The request-scoped platform context for the current request. - - Returns: - The store instance for the given hosting environment and context ID. - """ - - -# region Checkpoint persistence - - -class FoundryCheckpointStore: - """Checkpoint store backed by the `FoundryStateStore`.""" - - DEFAULT_ROOT_SCOPE = "checkpoints" - - def __init__( - self, - context_id: str, - platform_context: FoundryAgentRequestContext, - *, - allowed_checkpoint_types: list[str] | None = None, - scope: FoundryRequestScope | None = None, - ) -> None: - """Initialize a Foundry-scoped checkpoint store for the given context ID. - - Args: - context_id: A string that uniquely identifies the context for which the checkpoint store is scoped. - This can be used to isolate checkpoints for different workflow runs. - platform_context: The request-scoped platform context for the current request. - allowed_checkpoint_types: Additional types (beyond the built-in safe set - and framework types) that are permitted during checkpoint - deserialization. Each entry should be a ``"module:qualname"`` - string (e.g., ``"my_app.models:MyState"``). - scope: Trusted request scope for hosted sandbox isolation. - """ - if not context_id: - raise ValueError("context_id must be provided to initialize a FoundryCheckpointStore.") - - self.context_id = context_id - self.platform_context = platform_context - self._allowed_types: frozenset[str] = frozenset(allowed_checkpoint_types or []) - self._scope = scope - - async def _get_store(self) -> FoundryStateStore: - return await FoundryStateStore.get_or_create( - _store_name(self.DEFAULT_ROOT_SCOPE, self._scope, self.context_id), - user_isolation=True, - ) - - async def save(self, checkpoint: WorkflowCheckpoint) -> CheckpointID: - """Save a workflow checkpoint to the store. - - Args: - checkpoint: The workflow checkpoint to save. - - The checkpoint will be serialized to a dictionary and then encoded before being stored. - - Returns: - The ID of the saved checkpoint. - """ - from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value - - encoded_checkpoint = encode_checkpoint_value(checkpoint.to_dict()) - - store = await self._get_store() - async with store: - await store.set_item(checkpoint.checkpoint_id, encoded_checkpoint, call_id=self.platform_context.call_id) - return checkpoint.checkpoint_id - - async def _load_encoded(self, checkpoint_id: CheckpointID) -> Any: - store = await self._get_store() - async with store: - item = await store.get_item(checkpoint_id, call_id=self.platform_context.call_id) - if item is None: - raise WorkflowCheckpointException(f"No checkpoint found with ID {checkpoint_id}") - return item.value - - async def load(self, checkpoint_id: CheckpointID) -> WorkflowCheckpoint: - """Load a workflow checkpoint from the store. - - Args: - checkpoint_id: The ID of the checkpoint to load. - - Returns: - The loaded workflow checkpoint. - - Raises: - WorkflowCheckpointException: If no checkpoint is found with the given ID. - """ - from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value - - encoded = await self._load_encoded(checkpoint_id) - return WorkflowCheckpoint.from_dict(decode_checkpoint_value(encoded, allowed_types=self._allowed_types)) - - async def load_with_hash(self, checkpoint_id: CheckpointID) -> tuple[WorkflowCheckpoint, str]: - """Load a checkpoint and hash its exact persisted encoding. - - Pickle is not a canonical serialization across Python versions or repeated - encodes. Hashing the acknowledged storage value preserves exact tamper - detection without rejecting an unchanged checkpoint after deserialization. - """ - from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value - - encoded = await self._load_encoded(checkpoint_id) - checkpoint = WorkflowCheckpoint.from_dict(decode_checkpoint_value(encoded, allowed_types=self._allowed_types)) - return checkpoint, _encoded_checkpoint_hash(encoded) - - async def list_checkpoints(self, *, workflow_name: str) -> list[WorkflowCheckpoint]: - """List all workflow checkpoints for a given workflow name.""" - from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value - - store = await self._get_store() - checkpoints: list[WorkflowCheckpoint] = [] - after: str | None = None - async with store: - while True: - page = await store.list_keys(after=after, call_id=self.platform_context.call_id) - for item_key in page.keys: - item = await store.get_item(item_key.key, call_id=self.platform_context.call_id) - if item is None: - continue - checkpoint = WorkflowCheckpoint.from_dict( - decode_checkpoint_value(item.value, allowed_types=self._allowed_types) - ) - if checkpoint.workflow_name == workflow_name: - checkpoints.append(checkpoint) - if not page.has_more or page.last_id is None: - break - after = page.last_id - return checkpoints - - async def delete(self, checkpoint_id: CheckpointID) -> bool: - store = await self._get_store() - async with store: - deleted_item = await store.delete_item(checkpoint_id, call_id=self.platform_context.call_id) - return deleted_item.id is not None - - async def get_latest(self, *, workflow_name: str) -> WorkflowCheckpoint | None: - checkpoints = await self.list_checkpoints(workflow_name=workflow_name) - if not checkpoints: - return None - return _select_latest_checkpoint(checkpoints) - - async def list_checkpoint_ids(self, *, workflow_name: str) -> list[CheckpointID]: - checkpoints = await self.list_checkpoints(workflow_name=workflow_name) - return [checkpoint.checkpoint_id for checkpoint in checkpoints] - - -class CheckpointStoreProvider(ContextScopedStoreProvider[CheckpointStorage]): - """Provide workflow checkpoint store scoped to a context. - - A workflow context can contain multiple checkpoints. Scoping establishes the - collection boundary used to list checkpoints, restore the latest checkpoint, - and clean up older checkpoints without affecting another workflow context. - - This defaults to using the `FoundryCheckpointStore` in all environments. - """ - - def __init__(self, *, allowed_checkpoint_types: list[str] | None = None) -> None: - """Initialize the provider. - - Args: - allowed_checkpoint_types: Additional types (beyond the built-in safe set - and framework types) that are permitted during checkpoint - deserialization, forwarded to every store this provider creates. - Each entry should be a ``"module:qualname"`` string - (e.g., ``"my_app.models:MyState"``). - """ - self._allowed_checkpoint_types = allowed_checkpoint_types - - def validate_checkpoint_value(self, value: object) -> None: - """Verify a value can round-trip through this provider's restricted decoder. - - Native hosts use this before claiming a stored typed workflow turn so an - omitted application allowlist fails before executors or tools run. - """ - from agent_framework._workflows._checkpoint_encoding import ( - decode_checkpoint_value, - encode_checkpoint_value, - ) - - decode_checkpoint_value( - encode_checkpoint_value(value), - allowed_types=frozenset(self._allowed_checkpoint_types or ()), - ) - - def get_store( - self, - *, - config: AgentConfig, - context_id: str, - platform_context: FoundryAgentRequestContext, - ) -> CheckpointStorage: - """Get checkpoint store for the requested hosting environment.""" - if not context_id: - raise ValueError("context_id must be provided to get a checkpoint store.") - - return self.get_store_for_scope( - context_id=context_id, - platform_context=platform_context, - scope=_store_scope(config, platform_context), - ) - - def get_store_for_scope( - self, - *, - scope: FoundryRequestScope | None, - context_id: str, - platform_context: FoundryAgentRequestContext, - ) -> CheckpointStorage: - """Create a checkpoint store using a host's already validated scope. - - Native hosts use this after protocol-specific identity resolution. - ``get_store`` retains its strict configuration-based scope validation. - - Args: - scope: Trusted user and sandbox scope resolved by the protocol host. - context_id: Native workflow lineage's checkpoint collection. - platform_context: The current request's storage authorization context. - """ - return FoundryCheckpointStore( - context_id, - platform_context, - allowed_checkpoint_types=self._allowed_checkpoint_types, - scope=scope, - ) - - -# endregion Checkpoint persistence - -# region Function approval persistence - - -class FunctionApprovalStore(Protocol): - """Store for saving function approval requests.""" - - async def save_approval_request(self, approval_request_id: str, request: Content) -> None: - """Save a function approval request under the given ID.""" - ... - - async def load_approval_request(self, approval_request_id: str) -> Content: - """Load a function approval request by its ID.""" - ... - - -class FoundryFunctionApprovalStore: - """Function approval store backed by the `FoundryStateStore`.""" - - DEFAULT_ROOT_SCOPE = "function_approvals" - - def __init__( - self, - platform_context: FoundryAgentRequestContext, - *, - scope: FoundryRequestScope | None = None, - ) -> None: - self.platform_context = platform_context - self._scope = scope - - async def _get_store(self) -> FoundryStateStore: - return await FoundryStateStore.get_or_create( - _store_name(self.DEFAULT_ROOT_SCOPE, self._scope), - user_isolation=True, - ) - - async def save_approval_request(self, approval_request_id: str, request: Content) -> None: - store = await self._get_store() - async with store: - try: - await store.create_item(approval_request_id, request.to_dict(), call_id=self.platform_context.call_id) - except FoundryStorageConflictError as ex: - raise ValueError(f"Approval request with ID '{approval_request_id}' already exists.") from ex - - async def load_approval_request(self, approval_request_id: str) -> Content: - store = await self._get_store() - async with store: - item = await store.get_item(approval_request_id, call_id=self.platform_context.call_id) - if item is None: - raise KeyError(f"Approval request with ID '{approval_request_id}' does not exist.") - return Content.from_dict(item.value) - - -class FunctionApprovalStoreProvider(StoreProvider[FunctionApprovalStore]): - """Provide function approval store for the active hosting environment. - - This defaults to using the `FoundryFunctionApprovalStore` in all environments. - """ - - def get_store( - self, - *, - config: AgentConfig, - platform_context: FoundryAgentRequestContext, - ) -> FunctionApprovalStore: - """Get function approval store for the requested hosting environment.""" - return FoundryFunctionApprovalStore( - platform_context, - scope=_store_scope(config, platform_context), - ) - - -# endregion Function approval persistence - -# region Agent session persistence - - -class FoundryAgentSessionStore(SessionStore): - """Agent session store backed by the `FoundryStateStore`.""" - - DEFAULT_ROOT_SCOPE = "agent_sessions" - - def __init__( - self, - platform_context: FoundryAgentRequestContext, - *, - store_name: str | None = None, - scope: FoundryRequestScope | None = None, - ) -> None: - """Initialize session storage. - - Args: - platform_context: The request-scoped platform context. - store_name: Logical state-store name. Defaults to `agent_sessions`. - scope: Trusted Foundry request scope for sandbox-specific storage. - """ - if store_name is not None and (not isinstance(store_name, str) or not store_name.strip()): - raise ValueError("store_name must be a non-empty string") - self.platform_context = platform_context - self._store_name = self.DEFAULT_ROOT_SCOPE if store_name is None else store_name - self._scope = scope - self._etags: dict[str, str | None] = {} - - async def _get_store(self) -> FoundryStateStore: - return await FoundryStateStore.get_or_create( - _store_name(self._store_name, self._scope), - user_isolation=True, - ) - - async def get(self, session_id: str) -> AgentSession | None: - store = await self._get_store() - async with store: - item = await store.get_item(session_id, call_id=self.platform_context.call_id) - if item is None: - self._etags[session_id] = None - return None - if not item.etag: - raise RuntimeError("Foundry agent session storage returned a loaded session without an ETag.") - self._etags[session_id] = item.etag - return AgentSession.from_dict(item.value) - - async def set(self, session_id: str, session: AgentSession) -> None: - store = await self._get_store() - async with store: - try: - etag = self._etags.get(session_id) - if etag is not None: - result = await store.set_item( - session_id, session.to_dict(), if_match=etag, call_id=self.platform_context.call_id - ) - elif self._scope is not None or session_id in self._etags: - result = await store.create_item( - session_id, session.to_dict(), call_id=self.platform_context.call_id - ) - else: - result = await store.set_item(session_id, session.to_dict(), call_id=self.platform_context.call_id) - except (FoundryStorageConflictError, FoundryStoragePreconditionError) as exc: - raise RuntimeError("Another request advanced this agent session; reload before writing.") from exc - if not result.etag: - raise RuntimeError("Foundry agent session storage did not return an ETag after saving.") - self._etags[session_id] = result.etag - - async def delete(self, session_id: str) -> None: - store = await self._get_store() - async with store: - await store.delete_item(session_id, call_id=self.platform_context.call_id) - self._etags.pop(session_id, None) - - -class AgentSessionStoreProvider(StoreProvider[SessionStore]): - """Provide agent session store for the active hosting environment. - - This defaults to using the `FoundryAgentSessionStore` in all environments. - """ - - def __init__(self, *, store_name: str | None = None) -> None: - """Initialize the provider. - - Args: - store_name: Logical state-store name. Defaults to `agent_sessions`. - """ - if store_name is not None and (not isinstance(store_name, str) or not store_name.strip()): - raise ValueError("store_name must be a non-empty string") - self._store_name = store_name - - def _get_scoped_store( - self, platform_context: FoundryAgentRequestContext, scope: FoundryRequestScope | None - ) -> SessionStore: - return FoundryAgentSessionStore( - platform_context, - store_name=self._store_name, - scope=scope, - ) - - def get_store(self, *, config: AgentConfig, platform_context: FoundryAgentRequestContext) -> SessionStore: - """Get agent session store for the requested hosting environment.""" - return self._get_scoped_store(platform_context, _store_scope(config, platform_context)) - - -# endregion Agent session persistence diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py index d9a02501c56..d82a69c85e1 100644 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py +++ b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py @@ -4,17 +4,22 @@ import asyncio import inspect +import ipaddress +import json import logging import os +import re from collections.abc import Callable, Mapping, Sequence from contextlib import _AsyncGeneratorContextManager # pyright: ignore[reportPrivateUsage] from contextvars import ContextVar -from typing import TYPE_CHECKING, Any, Literal, cast -from urllib.parse import urlsplit +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any, Literal, TypeGuard, cast +from urllib.parse import urlparse, urlsplit import httpx2 as httpx from agent_framework import ( CachingSkillsSource, + Content, DeduplicatingSkillsSource, MCPSkillsSource, MCPStreamableHTTPTool, @@ -23,8 +28,9 @@ SkillsSourceContext, ) from agent_framework._telemetry import mark_feature_used -from agent_framework.exceptions import ToolExecutionException +from agent_framework.exceptions import ToolExecutionException, UserInputRequiredException from azure.ai.agentserver.core import get_request_context +from mcp import MCPError from mcp.types import Tool from typing_extensions import override @@ -34,12 +40,13 @@ from collections.abc import AsyncGenerator, Generator from datetime import timedelta - from agent_framework import Content, Skill + from agent_framework import Skill from agent_framework._mcp import MCPSpecificApproval from azure.core.credentials import AccessToken, TokenCredential from azure.core.credentials_async import AsyncTokenCredential from mcp.client.session import ClientSession - from mcp.types import Prompt, ReadResourceResult + from mcp.types import CallToolResult, Prompt, ReadResourceResult + from opentelemetry import context as otel_context from pydantic import AnyUrl AzureCredentialTypes = TokenCredential | AsyncTokenCredential @@ -59,6 +66,131 @@ "strict_toolbox_header_provider", default=False, ) +CONSENT_ERROR_CODE = -32006 +_OAUTH_HOST_PATTERN = re.compile(r"^[A-Za-z0-9._~-]+$") + + +@dataclass +class ConsentError: + name: str + consent_url: str + + +def _is_safe_oauth_consent_link(consent_link: object) -> TypeGuard[str]: + """Return whether a consent link is an absolute HTTPS URL safe to expose as an action.""" + if not isinstance(consent_link, str) or not consent_link: + return False + if any(char.isspace() or ord(char) < 0x20 or ord(char) == 0x7F for char in consent_link): + return False + + try: + parsed = urlparse(consent_link) + hostname = parsed.hostname + _ = parsed.port + except ValueError: + return False + + if parsed.scheme.lower() != "https" or not hostname or parsed.username is not None or parsed.password is not None: + return False + + if "%" in hostname: + return False + authority = parsed.netloc + if authority.startswith("["): + closing_bracket = authority.find("]") + if closing_bracket == -1: + return False + ipv6_literal = authority[1:closing_bracket] + suffix = authority[closing_bracket + 1 :] + if suffix and (not suffix.startswith(":") or not suffix[1:].isdigit()): + return False + try: + ipaddress.IPv6Address(ipv6_literal) + except ValueError: + return False + return True + if "[" in authority or "]" in authority or ":" in hostname: + return False + return _OAUTH_HOST_PATTERN.fullmatch(hostname) is not None + + +def _normalize_oauth_consent_origin(value: str, *, require_origin_only: bool) -> str: + if not _is_safe_oauth_consent_link(value): + raise ValueError("OAuth consent origins must be absolute HTTPS URLs without user information.") + + parsed = urlparse(value) + if require_origin_only and (parsed.path not in ("", "/") or parsed.params or parsed.query or parsed.fragment): + raise ValueError("OAuth consent origin entries must not include a path, query, parameters, or fragment.") + + hostname = parsed.hostname + if hostname is None: # pragma: no cover - validated above. + raise ValueError("OAuth consent origin must include a hostname.") + normalized_host = f"[{hostname.lower()}]" if ":" in hostname else hostname.lower() + port = parsed.port + return f"https://{normalized_host}" if port in (None, 443) else f"https://{normalized_host}:{port}" + + +def _normalize_allowed_oauth_consent_origins( # pyright: ignore[reportUnusedFunction] + allowed_origins: Sequence[str] | None, +) -> frozenset[str] | None: + """Normalize exact HTTPS origins; ``None`` retains the safe-link-only policy.""" + if allowed_origins is None: + return None + return frozenset(_normalize_oauth_consent_origin(origin, require_origin_only=True) for origin in allowed_origins) + + +def _is_allowed_oauth_consent_link( # pyright: ignore[reportUnusedFunction] + consent_link: object, allowed_origins: frozenset[str] | None +) -> TypeGuard[str]: + """Validate a link against safety rules and the optional normalized origin allowlist.""" + if not _is_safe_oauth_consent_link(consent_link): + return False + if allowed_origins is None: + return True + return _normalize_oauth_consent_origin(consent_link, require_origin_only=False) in allowed_origins + + +def consent_url_from_error(exc: BaseException) -> list[ConsentError] | None: + """Extract requirements only from a consent-only Foundry MCP gateway error or its wrapper.""" + inner = exc if isinstance(exc, MCPError) else next((arg for arg in exc.args if isinstance(arg, MCPError)), None) + if inner is None or inner.code != CONSENT_ERROR_CODE: + return None + start = inner.error.message.find("{") + if start == -1: + logger.warning("Foundry consent error does not contain a structured payload.") + return None + try: + payload: object = json.loads(inner.error.message[start:]) + except json.JSONDecodeError: + logger.warning("Foundry consent error contains invalid JSON.") + return None + if not isinstance(payload, dict): + logger.warning("Foundry consent error payload must be an object.") + return None + errors = cast(dict[str, object], payload).get("errors") + if not isinstance(errors, list): + logger.warning("Foundry consent error payload must contain an errors list.") + return None + + requirements: list[ConsentError] = [] + for raw_error in cast(list[object], errors): + if not isinstance(raw_error, dict): + return None + error = cast(dict[str, object], raw_error) + if error.get("type") not in ("mcp", "a2a_preview"): + return None + raw_details = error.get("error") + if not isinstance(raw_details, dict): + return None + details = cast(dict[str, object], raw_details) + if details.get("code") != "CONSENT_REQUIRED": + return None + name = error.get("name", "Unknown") + link = details.get("message") + if not isinstance(name, str) or not name or not isinstance(link, str): + return None + requirements.append(ConsentError(name=name, consent_url=link)) + return requirements or None def _build_toolbox_features_header(additional_features: str | None) -> str: @@ -265,6 +397,12 @@ class FoundryToolbox(MCPStreamableHTTPTool): The private metadata parser provides no security or compatibility guarantees when reused outside Foundry Toolbox; it is not a general-purpose MCP approval policy. + A validated invocation-time OAuth consent error raises + :class:`~agent_framework.exceptions.UserInputRequiredException`. The native + function loop returns a paired terminal result and consent action, retaining + completed sibling results without automatically retrying protected operations. + Responses hosting requires ``store=true`` for this continuation. + Like any MCP tool, the connection lifecycle is driven by the agent: the hosting server enters a factory-created agent for its request and closes its toolbox and owned HTTP client afterward. A long-lived Toolbox is also supported: it @@ -418,6 +556,29 @@ async def call_tool(self, tool_name: str, **kwargs: Any) -> str | list[Content]: finally: _strict_toolbox_header_provider.reset(token) + @override + async def _invoke_tool( + self, tool_name: str, kwargs: dict[str, Any], trace_context: otel_context.Context + ) -> CallToolResult: + try: + return await super()._invoke_tool(tool_name, kwargs, trace_context) + except MCPError as exc: + requirements = consent_url_from_error(exc) + if not requirements or any( + not _is_safe_oauth_consent_link(requirement.consent_url) for requirement in requirements + ): + raise + raise UserInputRequiredException( + contents=[ + Content.from_oauth_consent_request( + requirement.consent_url, + additional_properties={"openai": {"server_label": requirement.name}}, + ) + for requirement in requirements + ], + message="Tool invocation requires OAuth consent.", + ) from exc + @override async def _call_prompt_with_runtime_kwargs( self, diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_source.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_source.py deleted file mode 100644 index c22c216138d..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_source.py +++ /dev/null @@ -1,395 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -"""Built, request-owned workflow sources shared by the Foundry protocol hosts.""" - -from __future__ import annotations - -import inspect -import weakref -from collections import OrderedDict -from collections.abc import Awaitable, Callable, Iterator, Mapping, Sequence -from typing import Any, Generic, TypeAlias, TypeVar, cast - -from agent_framework import ( - Agent, - AgentContext, - AgentExecutor, - AgentMiddleware, - AgentResponse, - HistoryProvider, - InMemoryHistoryProvider, - RawAgent, - SupportsAgentRun, - Workflow, - WorkflowCheckpoint, - WorkflowExecutor, - WorkflowInvocationKwargs, - WorkflowRunState, -) - -from ._request import WorkflowTurn, validate_default_transport_options -from ._scope import FoundryRequestScope - -RequestT = TypeVar("RequestT") -WorkflowSource: TypeAlias = Workflow | Callable[[RequestT], Workflow | Awaitable[Workflow]] -_MAX_STRONG_RESOURCE_IDENTITIES = 1024 -_CLIENT_CONTROLS = frozenset({ - "additional_function_arguments", - "agent", - "background", - "call_id", - "checkpoint_id", - "checkpoint_storage", - "client_kwargs", - "continuation_token", - "conversation", - "conversation_id", - "extra_body", - "function_invocation_kwargs", - "fresh_factory", - "input", - "instructions", - "messages", - "middleware", - "previous_response_id", - "response_id", - "responses", - "service_session_id", - "session", - "session_id", - "agent_session_id", - "store", - "stream", - "tokenizer", - "tools", - "user", - "user_id", -}) - - -class _NativeAgentOptions(AgentMiddleware): - """Apply native workflow options at the existing agent boundary, not as client **kwargs.""" - - def __init__(self, scope: FoundryRequestScope) -> None: - self.scope = scope - - async def process(self, context: AgentContext, call_next: Callable[[], Awaitable[None]]) -> None: - options = context.client_kwargs.pop("options", {}) - if not isinstance(options, Mapping): - raise TypeError("Native workflow model options must be a mapping.") - options = cast(Mapping[str, Any], options) - if (_CLIENT_CONTROLS - {"store"}).intersection(options): - raise ValueError("Native workflow model options cannot override hosting or transport controls.") - if context.session is not None and context.session.service_session_id is not None: - raise RuntimeError("Native workflow checkpoints cannot resume a private downstream service session.") - effective = {**(context.options or {}), **options} - client = getattr(context.agent, "client", None) - if getattr(client, "STORES_BY_DEFAULT", None) is True: - effective["store"] = False - else: - if effective.pop("store", None) is True: - raise RuntimeError("The native workflow client cannot safely override a storing runtime option.") - context.options = effective - context.client_kwargs["additional_function_arguments"] = { - "session_id": self.scope.session_id, - "user_id": self.scope.user_id, - "call_id": self.scope.call_id, - } - - def check_result(result: AgentResponse[Any]) -> None: - if context.session is not None and context.session.service_session_id is not None: - raise RuntimeError("The native workflow client stored a service session despite store=False.") - if result.continuation_token is not None: - raise RuntimeError("Native workflows cannot checkpoint an unfinished provider background response.") - - if context.stream: - context.stream_result_hooks.append(check_result) - await call_next() - if not context.stream and isinstance(context.result, AgentResponse): - check_result(context.result) - - -def prepare_workflow_kwargs( - workflow: Workflow, - turn: WorkflowTurn[Any], - scope: FoundryRequestScope, - *, - options: Mapping[str, Any] | None = None, - stored: bool = True, - fresh_factory: bool = False, -) -> tuple[WorkflowInvocationKwargs, WorkflowInvocationKwargs]: - """Separate model options, explicit tool kwargs, and trusted current-call context. - - Agent executors use MAF checkpoint history, not downstream service storage. - Applications keep tool context in ``function_invocation_kwargs``; generation - options cannot become workflow, agent lifecycle, or storage controls. - ``Agent`` uses its existing middleware boundary without changing defaults. - Bare ``RawAgent`` requires ``fresh_factory`` from a validated resolver, - explicit non-storing defaults, and pre-materialized matching model overrides. - No nested options keyword is forwarded to a bare client's ``**kwargs``. - """ - client, _ = workflow._resolve_invocation_kwargs( # pyright: ignore[reportPrivateUsage] - turn.client_kwargs or {}, "client_kwargs" - ) - functions, _ = workflow._resolve_invocation_kwargs( # pyright: ignore[reportPrivateUsage] - turn.function_invocation_kwargs or {}, "function_invocation_kwargs" - ) - - def validate_values(value: Any) -> dict[str, Any]: - if not isinstance(value, Mapping) or any(not isinstance(key, str) for key in cast(Mapping[object, Any], value)): - raise TypeError("Workflow kwargs must have string-keyed mappings for global and executor values.") - return dict(cast(Mapping[str, Any], value)) - - global_client = validate_values(client.get("global_kwargs", {})) - specific_client = { - key: validate_values(value) for key, value in cast(dict[str, Any], client.get("executor_kwargs", {})).items() - } - for values in (global_client, *specific_client.values()): - if _CLIENT_CONTROLS.intersection(values): - raise ValueError("Workflow client kwargs cannot override hosting, agent, or transport controls.") - if "options" in values: - values["options"] = validate_values(values["options"]) - if _CLIENT_CONTROLS.intersection(values["options"]): - raise ValueError("Workflow model options cannot override hosting, agent, or transport controls.") - model_options = validate_values(options or {}) - if _CLIENT_CONTROLS.intersection(model_options): - raise ValueError("Workflow model options cannot override hosting, agent, or transport controls.") - - identity = {"session_id": scope.session_id, "user_id": scope.user_id, "call_id": scope.call_id} - global_client["additional_function_arguments"] = identity - global_model_options = global_client.pop("options", {}) - agents = list(workflow_agent_targets(workflow)) - for executor_id, agent_value in agents: - if not isinstance(agent_value, RawAgent): - raise TypeError( - "Native workflow custom agents must expose a verifiable client/storage contract. " - "Use Agent or a validated fresh RawAgent factory." - ) - agent = cast(RawAgent[Any], agent_value) - validate_default_transport_options(agent.default_options, allow_agent_store=False) - if any( - agent.default_options.get(name) is not None - for name in _CLIENT_CONTROLS - {"store", "instructions", "tools"} - ): - raise ValueError("Native workflow agents cannot have downstream continuation or hosting identity defaults.") - stores = getattr(agent.client, "STORES_BY_DEFAULT", None) - if not isinstance(stores, bool) or (not stores and agent.default_options.get("store") is True): - raise TypeError( - "Native workflow clients must declare storage support so hosting can disable inner storage." - ) - if not stored and any( - isinstance(provider, HistoryProvider) - and not isinstance(provider, InMemoryHistoryProvider) - and (provider.store_inputs or provider.store_outputs or provider.store_context_messages) - for provider in agent.context_providers - ): - raise ValueError("store=false cannot disable an external workflow HistoryProvider.") - values = specific_client.setdefault(executor_id, {}) - effective = {**global_model_options, **values.get("options", {}), **model_options} - values["additional_function_arguments"] = identity - if isinstance(agent_value, Agent): - if stores: - effective["store"] = False - values["options"] = effective - policies = [ - middleware for middleware in agent_value.middleware or () if isinstance(middleware, _NativeAgentOptions) - ] - if policies: - if len(policies) != 1 or policies[0].scope != scope: - raise RuntimeError("Native workflow option policy cannot be shared across request scopes.") - else: - agent_value.middleware = [*(agent_value.middleware or ()), _NativeAgentOptions(scope)] - else: - if not fresh_factory: - raise TypeError("Native RawAgent workflows require a validated request-aware fresh factory.") - if stores and agent.default_options.get("store") is not False: - raise ValueError("A storing native RawAgent client requires explicit factory default store=False.") - if any(agent.default_options.get(key) != value for key, value in effective.items()): - raise ValueError( - "RawAgent workflow generation overrides must be materialized in the fresh factory defaults." - ) - values.pop("options", None) - global_functions = validate_values(functions.get("global_kwargs", {})) - specific_functions = { - key: validate_values(value) for key, value in cast(dict[str, Any], functions.get("executor_kwargs", {})).items() - } - return ( - WorkflowInvocationKwargs(global_client, specific_client), - WorkflowInvocationKwargs(global_functions, specific_functions), - ) - - -def workflow_executors(workflow: Workflow) -> Iterator[AgentExecutor]: - """Walk agent executors, including those in built subworkflows.""" - for executor in workflow.get_executors_list(): - if isinstance(executor, AgentExecutor): - yield executor - elif isinstance(executor, WorkflowExecutor): - yield from workflow_executors(executor.workflow) - - -def workflow_agent_targets(workflow: Workflow) -> Iterator[tuple[str, SupportsAgentRun]]: - """Walk core and registry-backed executor agents with their run-kwargs routing IDs.""" - seen: set[tuple[str, int]] = set() - for graph in _workflow_graphs(workflow): - for executor in graph.get_executors_list(): - values: list[object] = [] - if isinstance(executor, AgentExecutor): - values.append(executor.agent) - registry = getattr(executor, "_agents", None) - if isinstance(registry, Mapping): - values.extend(cast(Mapping[object, object], registry).values()) - for value in values: - agent = value.agent if isinstance(value, AgentExecutor) else value - if not isinstance(agent, SupportsAgentRun): - raise TypeError("A workflow agent registry contains an unsupported agent value.") - key = (executor.id, id(agent)) - if key in seen: - continue - seen.add(key) - yield executor.id, agent - - -def workflow_agents(workflow: Workflow) -> Iterator[SupportsAgentRun]: - """Walk every unique agent owned by core or registry-backed executors.""" - seen: set[int] = set() - for _, agent in workflow_agent_targets(workflow): - if id(agent) in seen: - continue - seen.add(id(agent)) - yield agent - - -def _workflow_graphs(workflow: Workflow) -> Iterator[Workflow]: - yield workflow - for executor in workflow.get_executors_list(): - if isinstance(executor, WorkflowExecutor): - yield from _workflow_graphs(executor.workflow) - - -def validate_workflow_provider_state(workflow: Workflow, checkpoint: WorkflowCheckpoint | None = None) -> None: - """Refuse unexpected private service continuation before pairing or publishing native output.""" - if checkpoint is None: - for executor in workflow_executors(workflow): - if executor._session.service_session_id is not None: # pyright: ignore[reportPrivateUsage] - raise RuntimeError("The native workflow client stored a service session despite store=False.") - return - states = checkpoint.state.get("_executor_state", {}) - if not isinstance(states, Mapping): - raise ValueError("Invalid native workflow executor checkpoint state.") - states = cast(Mapping[str, Any], states) - for executor in workflow.get_executors_list(): - state = states.get(executor.id, {}) - if not isinstance(state, Mapping): - raise ValueError("Invalid native workflow executor checkpoint state.") - state = cast(Mapping[str, Any], state) - if isinstance(executor, AgentExecutor): - session = state.get("agent_session", {}) - if not isinstance(session, Mapping): - raise ValueError("Invalid native workflow agent checkpoint state.") - session = cast(Mapping[str, Any], session) - if session.get("service_session_id") is not None: - raise RuntimeError("The native workflow checkpoint contains unexpected private service continuation.") - elif isinstance(executor, WorkflowExecutor): - child = state.get("sub_workflow_checkpoint") - if child is not None: - if not isinstance(child, WorkflowCheckpoint): - raise ValueError("Invalid native subworkflow checkpoint state.") - validate_workflow_provider_state(executor.workflow, child) - - -def validate_workflow_source(source: object) -> None: - """Require a built workflow or a factory accepting the protocol's request object.""" - if isinstance(source, Workflow): - return - if not callable(source) or inspect.isclass(source): - raise TypeError("workflow must be a built Workflow or a request-aware factory returning a built Workflow.") - try: - inspect.signature(source).bind(object()) - except (TypeError, ValueError) as exc: - raise TypeError("A workflow factory must accept one request argument.") from exc - - -class WorkflowResolver(Generic[RequestT]): - """Reject reuse of mutable graphs and known request-bound resources. - - This is an ownership guard, not a clone operation or a proof that arbitrary - application globals are stateless. Factories must allocate their resources - without performing workflow or tool side effects. - """ - - def __init__(self, source: WorkflowSource[RequestT]) -> None: - validate_workflow_source(source) - self.source = source - self._owned: dict[int, weakref.ReferenceType[Any]] = {} - self._strong_owned: OrderedDict[int, object] = OrderedDict() - - @property - def is_factory(self) -> bool: - """Whether this source can produce a new graph for a later turn or recovery.""" - return not isinstance(self.source, Workflow) - - async def resolve(self, request: RequestT) -> Workflow: - """Resolve one fresh built graph without sharing executors, agents, clients, or providers.""" - if isinstance(self.source, Workflow): - workflow = self.source - else: - result = self.source(request) - workflow = await result if inspect.isawaitable(result) else result - if not isinstance(workflow, Workflow): - raise TypeError( - "The workflow factory must return a built Workflow, not a WorkflowBuilder or WorkflowAgent." - ) - resources: list[object] = [] - - def collect(graph: Workflow) -> None: - if ( - graph.status != WorkflowRunState.IDLE - or graph.get_last_checkpoint_id() is not None - or graph._runner.state.export_state() # pyright: ignore[reportPrivateUsage] - or graph._is_run_active() # pyright: ignore[reportPrivateUsage] - ): - raise RuntimeError("Native hosting requires a freshly built, unrun workflow.") - if graph._runner_context.has_checkpointing(): # pyright: ignore[reportPrivateUsage] - raise RuntimeError("Native hosting owns checkpoint storage; build the workflow without a store.") - resources.extend((graph, graph._runner_context)) # pyright: ignore[reportPrivateUsage] - for executor in graph.get_executors_list(): - resources.append(executor) - if isinstance(executor, WorkflowExecutor): - collect(executor.workflow) - - collect(workflow) - for agent_value in workflow_agents(workflow): - resources.append(agent_value) - if isinstance(agent_value, RawAgent): - agent = cast(RawAgent[Any], agent_value) - resources.append(agent.client) - resources.extend(agent.context_providers) - resources.extend(agent.mcp_tools) - tools = agent.default_options.get("tools", ()) - if isinstance(tools, Sequence): - resources.extend(tool for tool in cast(Sequence[object], tools) if not isinstance(tool, Mapping)) - unique_resources = {id(resource): resource for resource in resources} - for identifier, resource in unique_resources.items(): - previous = self._owned.get(id(resource)) - if (previous is not None and previous() is resource) or self._strong_owned.get(identifier) is resource: - raise RuntimeError( - "Native workflow requests cannot share workflows, executors, agents, clients, or providers. " - "Use a request-aware factory that creates fresh instances." - ) - for identifier, resource in unique_resources.items(): - try: - self._owned[identifier] = weakref.ref(resource, lambda ref, key=identifier: self._forget(key, ref)) - except TypeError: - # Some valid slotted protocol implementations omit __weakref__. - # Keep a bounded exact-identity fallback rather than rejecting them - # or retaining unbounded request resources for the host lifetime. - self._strong_owned[identifier] = resource - self._strong_owned.move_to_end(identifier) - while len(self._strong_owned) > _MAX_STRONG_RESOURCE_IDENTITIES: - self._strong_owned.popitem(last=False) - return workflow - - def _forget(self, key: int, reference: weakref.ReferenceType[Any]) -> None: - if self._owned.get(key) is reference: - del self._owned[key] diff --git a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py b/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py deleted file mode 100644 index 442eadb64e0..00000000000 --- a/python/packages/foundry_hosting/agent_framework_foundry_hosting/_workflow_state.py +++ /dev/null @@ -1,927 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -"""Exact native workflow pairs and linear reply authority within a trusted sandbox.""" - -from __future__ import annotations - -import hashlib -import json -import logging -import secrets -from collections.abc import AsyncGenerator, Awaitable, Callable, Mapping -from dataclasses import asdict, dataclass, replace -from typing import Any, Literal, cast - -from agent_framework import ( - AgentResponse, - AgentResponseUpdate, - ChatResponse, - ChatResponseUpdate, - CheckpointStorage, - Content, - Workflow, - WorkflowCheckpoint, - WorkflowEvent, - WorkflowInvocationKwargs, - WorkflowRunState, -) -from agent_framework._workflows._agent_executor import ( - _validate_computer_tool_result, # pyright: ignore[reportPrivateUsage] -) -from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value -from agent_framework._workflows._const import ( - RESOLVED_WORKFLOW_RUN_KWARGS_KEY, - ROUTED_WORKFLOW_RUN_KWARGS_KEY, - WORKFLOW_RUN_KWARGS_KEY, -) -from agent_framework._workflows._typing_utils import is_instance_of -from agent_framework._workflows._workflow import ( - _coerce_request_info_response, # pyright: ignore[reportPrivateUsage] -) -from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext -from azure.ai.agentserver.core.storage import ( - FoundryStateStore, - FoundryStorageConflictError, - FoundryStoragePreconditionError, -) - -from ._request import WorkflowTurn -from ._scope import FoundryRequestScope -from ._state_store import CheckpointStoreProvider, ContextScopedStoreProvider -from ._workflow_source import prepare_workflow_kwargs, validate_workflow_provider_state - -_CONFLICT = "Another request advanced this workflow. Reload its current response or start a fresh lineage." -_BLOCKED = "This workflow turn was interrupted or failed; start a fresh workflow lineage to avoid replaying effects." -logger = logging.getLogger(__name__) - - -class WorkflowConflictError(RuntimeError): - """A scoped workflow head, graph, or reply authority no longer permits this turn.""" - - -class WorkflowBlockedError(WorkflowConflictError): - """A failed or cancelled turn blocks its lineage to prevent uncertain effect replay.""" - - -def _key(kind: str, identifier: str) -> str: - if not isinstance(identifier, str) or not identifier: - raise ValueError("A non-empty workflow state identifier is required.") - return f"{kind}-{hashlib.sha256(identifier.encode('utf-8')).hexdigest()}" - - -def _json_object(value: Mapping[str, Any]) -> dict[str, Any]: - """Validate snapshots without arbitrary encoders or success-shaped string fallbacks.""" - if not isinstance(value, Mapping) or any(not isinstance(key, str) for key in value): - raise TypeError("A workflow output snapshot must be a JSON object.") - return cast(dict[str, Any], json.loads(json.dumps(dict(value), allow_nan=False))) - - -def _checkpoint_hash(checkpoint: WorkflowCheckpoint) -> str: - encoded = encode_checkpoint_value(checkpoint.to_dict()) - return hashlib.sha256(json.dumps(encoded, sort_keys=True, allow_nan=False).encode("utf-8")).hexdigest() - - -@dataclass(frozen=True) -class WorkflowBinding: - """The exact checkpoint for one outer response, not a sandbox or provider session.""" - - response_id: str - checkpoint_id: str | None - lineage_id: str - workflow_name: str - graph_hash: str - scope_key: str - checkpoint_hash: str | None = None - conversation_id: str | None = None - previous_response_id: str | None = None - - @classmethod - def from_value(cls, value: Any, scope: FoundryRequestScope) -> WorkflowBinding: - if not isinstance(value, dict): - raise ValueError("Invalid persisted workflow binding.") - value = cast(dict[str, Any], value) - if set(value) != set(cls.__dataclass_fields__): - raise ValueError("Invalid persisted workflow binding.") - for name in ("response_id", "lineage_id", "workflow_name", "graph_hash", "scope_key"): - if not isinstance(value[name], str) or not value[name]: - raise ValueError("Invalid persisted workflow binding.") - for name in ("checkpoint_id", "checkpoint_hash", "conversation_id", "previous_response_id"): - if value[name] is not None and (not isinstance(value[name], str) or not value[name]): - raise ValueError("Invalid persisted workflow binding.") - if (value["checkpoint_id"] is None) != (value["checkpoint_hash"] is None): - raise ValueError("Invalid persisted workflow checkpoint identity.") - binding = cls(**value) - if binding.scope_key != scope.storage_key: - raise PermissionError("The workflow state is not available in this trusted scope.") - return binding - - -@dataclass(frozen=True) -class WorkflowRecord: - """A per-response checkpoint/output pair retained independently of a later head.""" - - binding: WorkflowBinding - owner: str - status: Literal["active", "completed", "blocked"] - snapshot: dict[str, Any] | None = None - approvals: dict[str, str] | None = None - - @classmethod - def from_value(cls, value: Any, scope: FoundryRequestScope) -> WorkflowRecord: - if not isinstance(value, dict): - raise ValueError("Invalid persisted workflow response pair.") - value = cast(dict[str, Any], value) - if set(value) != set(cls.__dataclass_fields__): - raise ValueError("Invalid persisted workflow response pair.") - binding = WorkflowBinding.from_value(value["binding"], scope) - if ( - not isinstance(value["owner"], str) - or not value["owner"] - or value["status"] - not in ( - "active", - "completed", - "blocked", - ) - ): - raise ValueError("Invalid persisted workflow response pair.") - snapshot = value["snapshot"] - if snapshot is not None: - snapshot = _json_object(snapshot) - raw_approvals = value["approvals"] - approvals: dict[str, str] | None = None - if raw_approvals is not None: - if not isinstance(raw_approvals, dict): - raise ValueError("Invalid persisted workflow reply authority.") - approvals = {} - for wire_id, request_id in cast(dict[object, object], raw_approvals).items(): - if not isinstance(wire_id, str) or not wire_id or not isinstance(request_id, str) or not request_id: - raise ValueError("Invalid persisted workflow reply authority.") - approvals[wire_id] = request_id - if value["status"] == "completed" and (binding.checkpoint_id is None or snapshot is None): - raise ValueError("A completed workflow response requires an exact checkpoint/output pair.") - return cls(binding, value["owner"], value["status"], snapshot, approvals) - - -@dataclass(frozen=True) -class WorkflowHead: - """One CAS authority for a lineage, including its optional named conversation.""" - - lineage_id: str - conversation_id: str | None - binding: WorkflowBinding | None = None - response_id: str | None = None - owner: str | None = None - blocked: bool = False - - @classmethod - def from_value(cls, value: Any, scope: FoundryRequestScope) -> WorkflowHead: - if not isinstance(value, dict): - raise ValueError("Invalid persisted workflow head.") - value = cast(dict[str, Any], value) - if set(value) != set(cls.__dataclass_fields__): - raise ValueError("Invalid persisted workflow head.") - if not isinstance(value["lineage_id"], str) or not value["lineage_id"] or type(value["blocked"]) is not bool: - raise ValueError("Invalid persisted workflow head.") - for name in ("conversation_id", "response_id", "owner"): - if value[name] is not None and (not isinstance(value[name], str) or not value[name]): - raise ValueError("Invalid persisted workflow head.") - if (value["owner"] is None) != (value["response_id"] is None): - raise ValueError("Invalid persisted workflow claim.") - binding = WorkflowBinding.from_value(value["binding"], scope) if value["binding"] is not None else None - if binding is not None and ( - binding.lineage_id != value["lineage_id"] or binding.conversation_id != value["conversation_id"] - ): - raise ValueError("Invalid persisted workflow head binding.") - return cls( - value["lineage_id"], - value["conversation_id"], - binding, - value["response_id"], - value["owner"], - value["blocked"], - ) - - -class FoundryWorkflowBindingStore: - """Persist scoped per-response pairs and fenced linear workflow claims.""" - - def __init__(self, scope: FoundryRequestScope) -> None: - self.scope = scope - - async def _get_store(self) -> FoundryStateStore: - return await FoundryStateStore.get_or_create( - f"workflow_bindings/v2/{self.scope.storage_key}", user_isolation=True - ) - - async def get_response(self, response_id: str) -> tuple[WorkflowRecord | None, str | None]: - store = await self._get_store() - async with store: - item = await store.get_item(_key("response", response_id), call_id=self.scope.call_id) - if item is None: - return None, None - record = WorkflowRecord.from_value(item.value, self.scope) - if record.binding.response_id != response_id or not item.etag: - raise ValueError("Invalid persisted workflow response identity or ETag.") - return record, item.etag - - async def get_head(self, lineage_id: str, conversation_id: str | None) -> tuple[WorkflowHead | None, str | None]: - store = await self._get_store() - key = _key("conversation", conversation_id) if conversation_id is not None else _key("lineage", lineage_id) - async with store: - item = await store.get_item(key, call_id=self.scope.call_id) - if item is None: - return None, None - head = WorkflowHead.from_value(item.value, self.scope) - if ( - head.conversation_id != conversation_id - or (conversation_id is None and head.lineage_id != lineage_id) - or not item.etag - ): - raise ValueError("Invalid persisted workflow head identity or ETag.") - return head, item.etag - - async def save_response(self, record: WorkflowRecord, *, expected_etag: str | None) -> str: - if record.binding.scope_key != self.scope.storage_key: - raise PermissionError("Cannot write workflow state outside its trusted scope.") - return await self._write(_key("response", record.binding.response_id), asdict(record), expected_etag) - - async def save_head(self, head: WorkflowHead, *, expected_etag: str | None) -> str: - if head.binding is not None and head.binding.scope_key != self.scope.storage_key: - raise PermissionError("Cannot write workflow state outside its trusted scope.") - key = ( - _key("conversation", head.conversation_id) - if head.conversation_id is not None - else _key("lineage", head.lineage_id) - ) - return await self._write(key, asdict(head), expected_etag) - - async def delete_response(self, response_id: str, *, expected_etag: str) -> bool: - """Conditionally delete a superseded response record.""" - store = await self._get_store() - async with store: - try: - deleted = await store.delete_item( - _key("response", response_id), - if_match=expected_etag, - call_id=self.scope.call_id, - ) - except (FoundryStorageConflictError, FoundryStoragePreconditionError) as exc: - raise WorkflowConflictError(_CONFLICT) from exc - return deleted.id is not None - - async def _write(self, key: str, value: dict[str, Any], expected_etag: str | None) -> str: - store = await self._get_store() - async with store: - try: - if expected_etag is None: - result = await store.create_item(key, value, call_id=self.scope.call_id) - else: - result = await store.set_item(key, value, if_match=expected_etag, call_id=self.scope.call_id) - except (FoundryStorageConflictError, FoundryStoragePreconditionError) as exc: - raise WorkflowConflictError(_CONFLICT) from exc - if not result.etag: - raise RuntimeError("Workflow storage did not acknowledge a conditional write with an ETag.") - return result.etag - - -class _CheckpointWrites: - """Track acknowledged core saves, never selecting an unrelated latest checkpoint.""" - - def __init__(self, storage: CheckpointStorage, run: HostedWorkflowRun) -> None: - self.storage = storage - self.run = run - self.last_checkpoint: WorkflowCheckpoint | None = None - self.acknowledged_ids: set[str] = set() - - async def save(self, checkpoint: WorkflowCheckpoint) -> str: - await self.run.assert_claim() - validate_workflow_provider_state(self.run.workflow, checkpoint) - identifier = await self.storage.save(checkpoint) - if identifier != checkpoint.checkpoint_id: - raise RuntimeError("Workflow checkpoint storage acknowledged a different checkpoint.") - self.last_checkpoint = checkpoint - self.acknowledged_ids.add(identifier) - return identifier - - async def load(self, checkpoint_id: str) -> WorkflowCheckpoint: - checkpoint = await self.storage.load(checkpoint_id) - # Core checkpoint restoration follows run-kwargs setup. Retain fresh request - # kwargs rather than restoring the previous request's private call context. - fresh = self.run.workflow._runner.state.export_state() # pyright: ignore[reportPrivateUsage] - state = dict(checkpoint.state) - for key in (WORKFLOW_RUN_KWARGS_KEY, RESOLVED_WORKFLOW_RUN_KWARGS_KEY, ROUTED_WORKFLOW_RUN_KWARGS_KEY): - state[key] = fresh.get(key, {}) - return replace(checkpoint, state=state) - - async def load_persisted(self, checkpoint_id: str) -> tuple[WorkflowCheckpoint, str]: - """Load a checkpoint with a process-stable integrity hash when supported.""" - load_with_hash = getattr(cast(Any, self.storage), "load_with_hash", None) - if callable(load_with_hash): - loader = cast( - Callable[[str], Awaitable[tuple[WorkflowCheckpoint, str]]], - load_with_hash, - ) - checkpoint, checkpoint_hash = await loader(checkpoint_id) - if not isinstance(checkpoint, WorkflowCheckpoint) or not isinstance(checkpoint_hash, str): - raise TypeError("Checkpoint storage returned an invalid checkpoint integrity pair.") - return checkpoint, checkpoint_hash - checkpoint = await self.storage.load(checkpoint_id) - return checkpoint, _checkpoint_hash(checkpoint) - - async def list_checkpoints(self, *, workflow_name: str) -> list[WorkflowCheckpoint]: - return await self.storage.list_checkpoints(workflow_name=workflow_name) - - async def list_checkpoint_ids(self, *, workflow_name: str) -> list[str]: - return await self.storage.list_checkpoint_ids(workflow_name=workflow_name) - - async def get_latest(self, *, workflow_name: str) -> WorkflowCheckpoint | None: - return await self.storage.get_latest(workflow_name=workflow_name) - - async def delete(self, checkpoint_id: str) -> bool: - return await self.storage.delete(checkpoint_id) - - -class HostedWorkflowRun: - """A native turn with explicit validation, claim, pairing, commit, and abort boundaries. - - Protocol hosts own output encoding and lifecycle signals. Closing ``events`` - never commits a turn. A failed or cancelled claim is blocked rather than - releasing potentially consumed approvals or replaying external side effects. - """ - - def __init__( - self, - workflow: Workflow, - scope: FoundryRequestScope, - *, - stored: bool, - recovery: bool, - fresh_factory: bool, - reclaim_superseded: bool, - ) -> None: - self.workflow = workflow - self.scope = scope - self.stored = stored - self.recovery = recovery - self.fresh_factory = fresh_factory - self.reclaim_superseded = reclaim_superseded - self.store = FoundryWorkflowBindingStore(scope) - self.binding: WorkflowBinding - self._head: WorkflowHead | None = None - self._head_etag: str | None = None - self._record: WorkflowRecord | None = None - self._record_etag: str | None = None - self._storage: _CheckpointWrites | None = None - self._checkpoint: WorkflowCheckpoint | None = None - self._reply_approvals: dict[str, str] = {} - self._turn: WorkflowTurn[Any] | None = None - self._client_kwargs = WorkflowInvocationKwargs() - self._function_kwargs = WorkflowInvocationKwargs() - self._claimed = False - self._finalized = False - self.already_completed = False - - @classmethod - async def prepare( - cls, - workflow: Workflow, - *, - scope: FoundryRequestScope, - response_id: str, - config: AgentConfig, - platform_context: FoundryAgentRequestContext, - checkpoint_store_provider: ContextScopedStoreProvider[CheckpointStorage], - previous_response_id: str | None = None, - conversation_id: str | None = None, - lineage_id: str | None = None, - stored: bool = True, - recovery: bool = False, - replay_completed: bool = False, - fresh_factory: bool = False, - reclaim_superseded: bool = False, - ) -> HostedWorkflowRun: - """Read exact continuation state without claiming, consuming replies, or executing.""" - if not workflow.name or not workflow.graph_signature_hash: - raise ValueError("Native workflows require a stable, non-empty workflow name and graph identity.") - if previous_response_id is not None and conversation_id is not None: - raise ValueError("A workflow turn cannot combine previous_response_id and conversation.") - if not stored and (recovery or previous_response_id is not None or conversation_id is not None): - raise ValueError("store=false workflow requests cannot continue stored state; start a one-shot turn.") - run = cls( - workflow, - scope, - stored=stored, - recovery=recovery, - fresh_factory=fresh_factory, - reclaim_superseded=reclaim_superseded, - ) - run.binding = WorkflowBinding( - response_id=response_id, - checkpoint_id=None, - lineage_id=lineage_id or response_id, - workflow_name=workflow.name, - graph_hash=workflow.graph_signature_hash, - scope_key=scope.storage_key, - conversation_id=conversation_id, - previous_response_id=previous_response_id, - ) - if not stored: - return run - - own, own_etag = await run.store.get_response(response_id) - if not recovery and replay_completed and own is not None and own.status == "completed": - recovery = True - run.recovery = True - if recovery and own is not None: - run._record, run._record_etag = own, own_etag - run.binding = own.binding - run._reply_approvals = own.approvals or {} - elif own is not None: - raise WorkflowConflictError( - "This workflow response already exists; retrieve it instead of executing it again." - ) - - previous: WorkflowBinding | None = None - if previous_response_id is not None and (not recovery or own is None): - record, _ = await run.store.get_response(previous_response_id) - if record is None or record.status != "completed": - raise ValueError("The previous response has no completed workflow checkpoint in this trusted scope.") - previous = record.binding - if not recovery: - run._reply_approvals = record.approvals or {} - if previous.conversation_id is not None: - raise ValueError("A named workflow conversation cannot be forked through previous_response_id.") - run.binding = replace(run.binding, lineage_id=previous.lineage_id) - - head, etag = await run.store.get_head(run.binding.lineage_id, run.binding.conversation_id) - run._head, run._head_etag = head, etag - if recovery: - if head is None: - raise RuntimeError("The workflow recovery authority is unavailable; start a fresh lineage.") - if own is None: - if head.response_id != response_id or head.owner is None or head.blocked: - raise WorkflowConflictError(_CONFLICT) - run.binding = replace(run.binding, lineage_id=head.lineage_id) - blocked_record = WorkflowRecord(run.binding, head.owner, "blocked") - await run.store.save_response(blocked_record, expected_etag=None) - await run.store.save_head(replace(head, blocked=True), expected_etag=etag) - raise WorkflowBlockedError(_BLOCKED) - if own.status == "blocked": - if head.response_id == response_id and head.owner == own.owner and not head.blocked: - await run.store.save_head(replace(head, blocked=True), expected_etag=etag) - raise WorkflowBlockedError(_BLOCKED) - if own.binding.checkpoint_id is None or own.snapshot is None: - if head.response_id != response_id or head.owner != own.owner or head.blocked: - raise WorkflowConflictError(_CONFLICT) - blocked_record = replace(own, status="blocked") - await run.store.save_response(blocked_record, expected_etag=own_etag) - await run.store.save_head(replace(head, blocked=True), expected_etag=etag) - raise WorkflowBlockedError(_BLOCKED) - if own.status == "completed": - current = head.binding - visited: set[str] = set() - while current is not None and current != own.binding: - if current.response_id in visited or current.previous_response_id is None: - current = None - break - visited.add(current.response_id) - parent, _ = await run.store.get_response(current.previous_response_id) - current = parent.binding if parent is not None and parent.status == "completed" else None - if current is not None: - run.already_completed = True - elif head.response_id != response_id or head.blocked: - raise WorkflowConflictError(_CONFLICT) - else: - run._finalized = True - elif head.response_id != response_id or head.owner != own.owner or head.blocked: - raise WorkflowConflictError(_CONFLICT) - previous = run.binding - elif head is not None: - if head.blocked: - raise WorkflowBlockedError(_BLOCKED) - if head.response_id is not None: - active_record, _ = await run.store.get_response(head.response_id) - if active_record is None and head.owner is not None: - interrupted_binding = WorkflowBinding( - response_id=head.response_id, - checkpoint_id=None, - lineage_id=head.lineage_id, - workflow_name=workflow.name, - graph_hash=workflow.graph_signature_hash, - scope_key=scope.storage_key, - conversation_id=head.conversation_id, - previous_response_id=head.binding.response_id if head.binding is not None else None, - ) - await run.store.save_response( - WorkflowRecord(interrupted_binding, head.owner, "blocked"), - expected_etag=None, - ) - await run.store.save_head(replace(head, blocked=True), expected_etag=etag) - raise WorkflowBlockedError(_BLOCKED) - raise WorkflowConflictError( - "This workflow has an in-flight turn. Wait for it or start a fresh lineage." - ) - if previous is not None and head.binding != previous: - raise WorkflowConflictError("This workflow continuation is stale or forked; use its current response.") - previous = head.binding - if previous is not None: - record, _ = await run.store.get_response(previous.response_id) - if record is None or record.status != "completed" or record.binding != previous: - raise ValueError("The workflow head has no exact completed response binding.") - run._reply_approvals = record.approvals or {} - run.binding = replace( - run.binding, - lineage_id=head.lineage_id, - previous_response_id=previous.response_id if previous is not None else None, - ) - elif previous is not None: - raise ValueError("The workflow lineage authority is unavailable in this trusted scope.") - - context_id = _key("native", f"{scope.storage_key}:{run.binding.lineage_id}") - storage = ( - checkpoint_store_provider.get_store_for_scope( - scope=scope, context_id=context_id, platform_context=platform_context - ) - if isinstance(checkpoint_store_provider, CheckpointStoreProvider) - else checkpoint_store_provider.get_store( - config=config, context_id=context_id, platform_context=platform_context - ) - ) - run._storage = _CheckpointWrites(storage, run) - if previous is not None: - if previous.workflow_name != workflow.name or previous.graph_hash != workflow.graph_signature_hash: - raise WorkflowConflictError( - "The stored workflow graph is incompatible. Rebuild its original graph or start a fresh lineage." - ) - if previous.checkpoint_id is None: - raise ValueError("The workflow response has no acknowledged checkpoint.") - checkpoint, checkpoint_hash = await run._storage.load_persisted(previous.checkpoint_id) - if ( - checkpoint.checkpoint_id != previous.checkpoint_id - or checkpoint.workflow_name != workflow.name - or checkpoint.graph_signature_hash != workflow.graph_signature_hash - or checkpoint_hash != previous.checkpoint_hash - ): - raise ValueError("The exact workflow checkpoint does not match its response binding.") - run._checkpoint = checkpoint - validate_workflow_provider_state(workflow, checkpoint) - return run - - @property - def snapshot(self) -> dict[str, Any] | None: - """The exact same-response recovery snapshot, independent of the current lineage head.""" - return self._record.snapshot if self._record is not None else None - - @property - def approvals(self) -> Mapping[str, str]: - """Caller-facing approval aliases bound to the loaded response's exact checkpoint.""" - return self._reply_approvals - - @property - def has_completed_output(self) -> bool: - """Whether recovery can deliver already finalized output without rerunning executors.""" - return self._record is not None and self._record.status == "completed" - - @property - def pending_requests(self) -> Mapping[str, WorkflowEvent[Any]]: - """Pending request authority from the exact acknowledged checkpoint.""" - checkpoint = self.current_checkpoint - return checkpoint.pending_request_info_events if checkpoint is not None else {} - - @property - def current_checkpoint(self) -> WorkflowCheckpoint | None: - """The last acknowledged checkpoint from this run, or its exact restoration checkpoint.""" - if self._storage is not None and self._storage.last_checkpoint is not None: - return self._storage.last_checkpoint - return self._checkpoint - - @property - def checkpoint_id(self) -> str | None: - """The exact acknowledged checkpoint ID, never a latest-checkpoint query.""" - checkpoint = self.current_checkpoint - return checkpoint.checkpoint_id if checkpoint is not None else None - - def recovery_turn(self) -> WorkflowTurn[Any]: - """Restore application kwargs without re-parsing already consumed approval input. - - The checkpoint, rather than a new start input or replayed reply, is the - recovery action. Host-injected identity and storage controls are replaced - by ``prepare_workflow_kwargs`` for the current trusted request. - """ - if not self.recovery or self._checkpoint is None: - raise RuntimeError("A recovery turn requires an exact restored checkpoint.") - resolved = self._checkpoint.state.get(RESOLVED_WORKFLOW_RUN_KWARGS_KEY, {}) - if not isinstance(resolved, Mapping): - raise ValueError("Invalid persisted workflow invocation kwargs.") - resolved = cast(Mapping[str, Any], resolved) - - def invocation_kwargs(name: str) -> WorkflowInvocationKwargs: - values = resolved.get(name, {}) - if not isinstance(values, Mapping): - raise ValueError("Invalid persisted workflow invocation kwargs.") - values = cast(Mapping[str, Any], values) - global_kwargs = dict(values.get("global_kwargs", {})) - executor_kwargs = {key: dict(value) for key, value in values.get("executor_kwargs", {}).items()} - if name == "client_kwargs": - for kwargs in (global_kwargs, *executor_kwargs.values()): - kwargs.pop("additional_function_arguments", None) - if "options" in kwargs: - kwargs["options"] = dict(kwargs["options"]) - kwargs["options"].pop("store", None) - return WorkflowInvocationKwargs(global_kwargs, executor_kwargs) - - return WorkflowTurn( - input=self._checkpoint, - client_kwargs=invocation_kwargs("client_kwargs"), - function_invocation_kwargs=invocation_kwargs("function_invocation_kwargs"), - ) - - def validate_turn(self, turn: WorkflowTurn[Any]) -> WorkflowTurn[Any]: - """Validate all input/replies and existing workflow kwargs before claiming authority.""" - if not isinstance(turn, WorkflowTurn): - raise TypeError("The workflow parser must return WorkflowTurn.") - self._client_kwargs, self._function_kwargs = prepare_workflow_kwargs( - self.workflow, - turn, - self.scope, - stored=self.stored, - fresh_factory=self.fresh_factory, - ) - if self.recovery: - self._turn = turn - return turn - pending = self.pending_requests - if turn.responses is not None: - if not self.stored or self._checkpoint is None: - raise ValueError("Workflow replies require a pending stored checkpoint.") - if set(turn.responses) != set(pending) or not pending: - raise ValueError("Workflow replies must answer exactly the complete pending request batch.") - replies: dict[str, Any] = {} - reply_ids: set[tuple[str, str]] = set() - for request_id, reply in turn.responses.items(): - event = pending[request_id] - try: - reply = _coerce_request_info_response(reply, event.response_type, request_id) - except (TypeError, ValueError) as exc: - raise ValueError("A workflow reply does not match its pending request type.") from exc - if isinstance(event.data, Content): - request = event.data - if isinstance(reply, Content): - identifier = reply.id if reply.type == "function_approval_response" else reply.call_id - if identifier is not None: - key = (reply.type, identifier) - if key in reply_ids: - raise ValueError("Duplicate workflow reply authority is not allowed.") - reply_ids.add(key) - if request.type == "function_approval_request" and ( - not isinstance(reply, Content) - or reply.type != "function_approval_response" - or reply.id != request.id - or type(reply.approved) is not bool - or reply.function_call is None - or request.function_call is None - or reply.function_call.to_dict() != request.function_call.to_dict() - ): - raise ValueError("The approval decision does not match its exact pending function call.") - if request.type == "function_call" and ( - not isinstance(reply, Content) - or reply.type != "function_result" - or reply.call_id != request.call_id - ): - raise ValueError("The function result does not match its pending call.") - if request.type == "computer_tool_call": - if not isinstance(reply, Content): - raise ValueError("A computer reply must be typed Content.") - _validate_computer_tool_result(request, reply) - replies[request_id] = reply - turn = replace(turn, responses=replies) - else: - if pending: - raise ValueError("This workflow is awaiting replies; do not replace them with new input.") - if self._checkpoint is not None and any(self._checkpoint.messages.values()): - raise ValueError("The workflow is still in flight; new input cannot replace its checkpoint.") - if not any(is_instance_of(turn.input, input_type) for input_type in self.workflow.input_types): - raise ValueError("The workflow parser's input does not match the start executor's type.") - self._turn = turn - return turn - - async def claim(self) -> None: - """Fence the exact validated turn before any executor or response handler runs.""" - if self._turn is None: - raise RuntimeError("Validate the workflow turn before claiming it.") - if self._claimed or self.already_completed: - return - if not self.stored: - self._claimed = True - return - owner = secrets.token_hex(16) - head = replace( - self._head or WorkflowHead(self.binding.lineage_id, self.binding.conversation_id), - response_id=self.binding.response_id, - owner=owner, - ) - self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) - self._head = head - self._claimed = True - record = ( - replace(self._record, owner=owner) - if self.recovery and self._record is not None - else WorkflowRecord(self.binding, owner, "active") - ) - self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) - self._record = record - await self._reclaim_completed_ancestors() - - async def assert_claim(self) -> None: - if not self._claimed: - raise RuntimeError("The workflow turn has not been claimed.") - if not self.stored: - return - head, etag = await self.store.get_head(self.binding.lineage_id, self.binding.conversation_id) - if head != self._head or etag != self._head_etag or head is None or head.blocked: - raise WorkflowConflictError(_CONFLICT) - - async def events( - self, - turn: WorkflowTurn[Any], - *, - client_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None, - function_invocation_kwargs: WorkflowInvocationKwargs | Mapping[str, Any] | None = None, - ) -> AsyncGenerator[WorkflowEvent[Any]]: - """Execute core events, with explicit caller-owned commit and cancellation boundaries.""" - if self.already_completed or self.has_completed_output: - return - await self.assert_claim() - if self._turn is None: - raise RuntimeError("The workflow turn has not been validated.") - turn = self._turn - if self._checkpoint is not None and not self.recovery: - await self.workflow._runner.restore_checkpoint(self._checkpoint) # pyright: ignore[reportPrivateUsage] - stream = self.workflow.run( - None if self.recovery else turn.input, - responses=None if self.recovery else turn.responses, - checkpoint_id=self._checkpoint.checkpoint_id if self.recovery and self._checkpoint is not None else None, - checkpoint_storage=self._storage, - stream=True, - client_kwargs=client_kwargs if client_kwargs is not None else self._client_kwargs, - function_invocation_kwargs=( - function_invocation_kwargs if function_invocation_kwargs is not None else self._function_kwargs - ), - ) - try: - async for event in stream: - data = event.data - if ( - isinstance(data, (AgentResponse, AgentResponseUpdate, ChatResponse, ChatResponseUpdate)) - and data.continuation_token is not None - ): - raise RuntimeError("Native workflows cannot checkpoint unfinished provider background output.") - if event.type == "superstep_completed": - validate_workflow_provider_state(self.workflow) - if event.type == "request_info" and not self.stored: - raise ValueError("Approval and user-input continuation requires store=true.") - yield event - result = await stream.get_final_response() - if result.get_final_state() not in (WorkflowRunState.IDLE, WorkflowRunState.IDLE_WITH_PENDING_REQUESTS): - raise RuntimeError("The workflow did not finalize successfully.") - validate_workflow_provider_state(self.workflow) - self._finalized = True - finally: - await stream.close() - - async def stage( - self, - snapshot: Mapping[str, Any], - *, - approvals: Mapping[str, str] | None = None, - checkpoint_id: str | None = None, - ) -> None: - """CAS-pair output with an acknowledged core checkpoint before publishing its events.""" - if not self.stored: - return - snapshot_value = _json_object(snapshot) - checkpoint, checkpoint_hash = await self._get_checkpoint_with_hash(checkpoint_id or self.checkpoint_id) - if checkpoint is None: - raise RuntimeError("Output cannot be paired before a core checkpoint is acknowledged.") - if approvals is not None and any( - request_id not in checkpoint.pending_request_info_events for request_id in approvals.values() - ): - raise ValueError("Approval authority must belong to this exact pending checkpoint.") - await self.assert_claim() - if self._record is None: - raise RuntimeError("The workflow response pair has not been claimed.") - binding = replace(self.binding, checkpoint_id=checkpoint.checkpoint_id, checkpoint_hash=checkpoint_hash) - pending_approvals = { - wire_id: request_id - for wire_id, request_id in (approvals if approvals is not None else self._record.approvals or {}).items() - if request_id in checkpoint.pending_request_info_events - } - record = replace( - self._record, - binding=binding, - snapshot=snapshot_value, - approvals=pending_approvals, - ) - self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) - self.binding, self._record = binding, record - await self.assert_claim() - - async def get_checkpoint(self, checkpoint_id: str | None) -> WorkflowCheckpoint | None: - """Read a checkpoint acknowledged in this run, including a producer-stamped stream boundary.""" - checkpoint, _ = await self._get_checkpoint_with_hash(checkpoint_id) - return checkpoint - - async def _get_checkpoint_with_hash( - self, checkpoint_id: str | None - ) -> tuple[WorkflowCheckpoint | None, str | None]: - if checkpoint_id is None: - return None, None - if self._storage is None or ( - checkpoint_id not in self._storage.acknowledged_ids - and (self._checkpoint is None or checkpoint_id != self._checkpoint.checkpoint_id) - ): - raise ValueError("Output pairing requires a checkpoint acknowledged by this exact workflow turn.") - checkpoint, checkpoint_hash = await self._storage.load_persisted(checkpoint_id) - if ( - checkpoint.checkpoint_id != checkpoint_id - or checkpoint.workflow_name != self.workflow.name - or checkpoint.graph_signature_hash != self.workflow.graph_signature_hash - ): - raise ValueError("The acknowledged workflow checkpoint identity changed.") - validate_workflow_provider_state(self.workflow, checkpoint) - return checkpoint, checkpoint_hash - - async def commit(self, snapshot: Mapping[str, Any] | None = None) -> None: - """Commit only fully finalized, encoded output, before protocol success or done.""" - if self.already_completed: - return - if not self._finalized: - raise RuntimeError("A workflow cannot commit before successful core finalization.") - if not self.stored: - return - if snapshot is not None: - await self.stage(snapshot) - await self.assert_claim() - if self._record is None or self._record.snapshot is None or self.binding.checkpoint_id != self.checkpoint_id: - raise RuntimeError("The final workflow output has not been paired with its exact checkpoint.") - record = replace(self._record, status="completed") - self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) - self._record = record - head = WorkflowHead(self.binding.lineage_id, self.binding.conversation_id, self.binding) - self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) - self._head = head - self._claimed = False - - async def _reclaim_completed_ancestors(self) -> None: - """Best-effort reclaim after the new head is durably committed. - - Cleanup is oldest-first. If a deletion fails, newer records retain the - chain needed for a later successful commit to retry cleanup. - """ - if not self.reclaim_superseded or self._storage is None: - return - response_id = self.binding.previous_response_id - records: list[tuple[WorkflowRecord, str]] = [] - visited: set[str] = set() - try: - current_checkpoint_id = self.binding.checkpoint_id - checkpoint_ids = await self._storage.storage.list_checkpoint_ids(workflow_name=self.binding.workflow_name) - for checkpoint_id in checkpoint_ids: - if current_checkpoint_id is None or checkpoint_id != current_checkpoint_id: - await self._storage.storage.delete(checkpoint_id) - while response_id is not None: - if response_id in visited: - raise ValueError("Invalid cycle in superseded workflow response lineage.") - visited.add(response_id) - record, etag = await self.store.get_response(response_id) - if record is None or etag is None: - break - if ( - record.status != "completed" - or record.binding.scope_key != self.scope.storage_key - or record.binding.lineage_id != self.binding.lineage_id - or record.binding.conversation_id != self.binding.conversation_id - ): - break - records.append((record, etag)) - response_id = record.binding.previous_response_id - for record, etag in reversed(records): - await self.store.delete_response(record.binding.response_id, expected_etag=etag) - except Exception: - logger.exception("Failed to reclaim superseded native workflow state; a later commit can retry cleanup") - - async def abort(self) -> None: - """Block interrupted/failed authority instead of permitting unsafe reply or effect replay.""" - if not self.stored or not self._claimed or self.already_completed: - return - await self.assert_claim() - if self._record is not None: - record = replace(self._record, status="blocked") - self._record_etag = await self.store.save_response(record, expected_etag=self._record_etag) - self._record = record - if self._head is not None: - head = replace(self._head, blocked=True) - self._head_etag = await self.store.save_head(head, expected_etag=self._head_etag) - self._head = head - self._claimed = False diff --git a/python/packages/foundry_hosting/tests/conftest.py b/python/packages/foundry_hosting/tests/conftest.py index abf0234046a..daa8cb51255 100644 --- a/python/packages/foundry_hosting/tests/conftest.py +++ b/python/packages/foundry_hosting/tests/conftest.py @@ -2,12 +2,243 @@ """Fixtures for Foundry Hosting tests.""" -from collections.abc import Iterator +from __future__ import annotations + +import asyncio +import json +from collections.abc import AsyncIterator, Awaitable, Callable, Iterator, Mapping, Sequence +from functools import partial from pathlib import Path from tempfile import TemporaryDirectory +from typing import TYPE_CHECKING, Any import pytest +if TYPE_CHECKING: + from unittest.mock import MagicMock + + from agent_framework import Agent, AgentResponseUpdate, AgentSession, Content, Message + from agent_framework.openai import OpenAIChatClient + + from agent_framework_foundry_hosting import ( + InvocationsHostServer, + InvocationsWorkflowHostServer, + ResponsesHostServer, + ResponsesWorkflowHostServer, + ) + + +@pytest.fixture +def approval_session() -> Callable[[Content], AgentSession]: + from agent_framework import AgentSession + from agent_framework._tools import _store_pending_approval_requests + + def create(request: Content) -> AgentSession: + session = AgentSession() + _store_pending_approval_requests(session, [request]) + return session + + return create + + +@pytest.fixture +def deferred_approval_agent( + approval_session: Callable[[Content], AgentSession], +) -> Callable[ + [Sequence[AgentResponseUpdate], bool], + tuple[MagicMock, list[AgentResponseUpdate], asyncio.Event], +]: + from unittest.mock import MagicMock + + from agent_framework import AgentResponse, AgentResponseUpdate, Content, ResponseStream, SupportsAgentRun + + def create( + tail: Sequence[AgentResponseUpdate], repeat: bool + ) -> tuple[MagicMock, list[AgentResponseUpdate], asyncio.Event]: + approval = Content.from_function_approval_request( + "deferred-approval", + Content.from_function_call("deferred-call", "delete_file", arguments={}, hosted=True), + ) + updates = [AgentResponseUpdate(role="assistant", contents=[approval]), *tail] + closed = asyncio.Event() + agent = MagicMock(spec=SupportsAgentRun) + agent.id = "deferred-approval-agent" + agent.name = "Deferred approval agent" + agent.description = "Produce approval output with observable stream cleanup." + agent.context_providers = [] + agent.create_session.side_effect = lambda **kwargs: approval_session(approval) + + async def generate() -> AsyncIterator[AgentResponseUpdate]: + try: + for update in updates: + yield update + if repeat: + empty = AgentResponseUpdate(role="assistant") + while True: + yield empty + finally: + closed.set() + + def run(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate, AgentResponse]: + return ResponseStream(generate(), finalizer=AgentResponse.from_updates) + + agent.run.side_effect = run + return agent, updates, closed + + return create + + +@pytest.fixture +def native_approval_agent(request: pytest.FixtureRequest) -> tuple[Agent, list[str], list[list[Message]]]: + from agent_framework import ( + Agent, + BaseChatClient, + ChatOptions, + ChatResponse, + ChatResponseUpdate, + Content, + FunctionInvocationLayer, + ResponseStream, + tool, + ) + + executions: list[str] = [] + calls: list[list[Message]] = [] + paths: tuple[str, ...] = getattr(request, "param", ("/safe",)) + + @tool(approval_mode="always_require") + def delete_file(path: str) -> str: + """Record a protected operation without accessing the filesystem.""" + executions.append(path) + return "deleted" + + class ApprovalClient(FunctionInvocationLayer[ChatOptions], BaseChatClient[ChatOptions]): + def _inner_get_response( + self, *, messages: Sequence[Message], stream: bool, options: Mapping[str, Any], **kwargs: Any + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + calls.append(list(messages)) + contents = ( + [ + Content.from_function_call( + f"provider-call-{index}" if index else "provider-call", + "delete_file", + arguments={"path": path}, + id=f"native-call-{index}" if index else "native-call", + ) + for index, path in enumerate(paths) + ] + if len(calls) == 1 + else [Content.from_text("done")] + ) + + async def updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate(role="assistant", contents=contents) + + return ResponseStream(updates(), finalizer=ChatResponse.from_updates) + + return Agent[ChatOptions](client=ApprovalClient(), tools=[delete_file]), executions, calls + + +@pytest.fixture +async def hosted_approval_client( + request: pytest.FixtureRequest, +) -> AsyncIterator[tuple[OpenAIChatClient, list[dict[str, Any]]]]: + from importlib import import_module + + from agent_framework.openai import OpenAIChatClient, OpenAIChatOptions + from openai import AsyncOpenAI, DefaultAsyncHttpxClient + + calls: list[dict[str, Any]] = [] + fail_after_approval = getattr(request, "param", None) == "failure_after_approval" + # OpenAI supports both httpx and httpx2; match the SDK-selected transport. + sdk_httpx = import_module(DefaultAsyncHttpxClient.__mro__[1].__module__.partition(".")[0]) + + def respond(request: Any) -> Any: + assert request.method == "POST" + assert request.url.path == "/v1/responses" + payload = json.loads(request.content) + calls.append(payload) + stored = payload.get("store", True) + response: dict[str, Any] = { + "id": f"resp_hosted_{len(calls)}", + "object": "response", + "created_at": 1_700_000_000, + "model": "test-model", + "status": "in_progress", + "output": [], + "error": None, + "incomplete_details": None, + "instructions": None, + "metadata": {}, + "parallel_tool_calls": False, + "temperature": 1.0, + "tool_choice": "auto", + "tools": [], + "top_p": 1.0, + "usage": None, + "store": stored, + "background": False, + "conversation": {"id": "conv_hosted"} if stored else None, + } + output: list[dict[str, Any]] = ( + [ + { + "type": "mcp_approval_request", + "id": "provider-approval", + "arguments": '{"path": "/safe"}', + "name": "delete_file", + "server_label": "hosted-tools", + } + ] + if len(calls) == 1 + else [ + { + "type": "message", + "id": f"msg_hosted_{len(calls)}", + "role": "assistant", + "status": "completed", + "content": [{"type": "output_text", "text": "done", "annotations": [], "logprobs": []}], + } + ] + ) + events: list[dict[str, Any]] = [ + {"type": "response.created", "response": response}, + { + "type": "response.output_item.added", + "output_index": 0, + "item": {**output[0], "content": []} if output[0]["type"] == "message" else output[0], + }, + ] + if output[0]["type"] == "message": + events.append({ + "type": "response.output_text.delta", + "item_id": output[0]["id"], + "output_index": 0, + "content_index": 0, + "delta": "done", + "logprobs": [], + }) + if fail_after_approval and len(calls) == 1: + events.append({"type": "error", "code": "server_error", "message": "provider failed", "param": None}) + else: + events.append({"type": "response.output_item.done", "output_index": 0, "item": output[0]}) + events.append({ + "type": "response.completed", + "response": {**response, "status": "completed", "output": output}, + }) + for index, event in enumerate(events): + event["sequence_number"] = index + body = "".join(f"event: {event['type']}\ndata: {json.dumps(event)}\n\n" for event in events) + return sdk_httpx.Response(200, headers={"content-type": "text/event-stream"}, text=body + "data: [DONE]\n\n") + + async with AsyncOpenAI( + api_key="test-key", + base_url="https://provider.test/v1", + max_retries=0, + http_client=DefaultAsyncHttpxClient(transport=sdk_httpx.MockTransport(respond)), + ) as api_client: + yield OpenAIChatClient[OpenAIChatOptions](model="test-model", async_client=api_client), calls + @pytest.fixture(autouse=True) def disable_agentserver_external_telemetry(monkeypatch: pytest.MonkeyPatch) -> None: @@ -22,3 +253,36 @@ def isolate_local_agentserver_state_root(monkeypatch: pytest.MonkeyPatch) -> Ite with TemporaryDirectory(prefix="agentserver-test-") as directory: monkeypatch.setenv("AGENTSERVER_STATE_ROOT", str(Path(directory) / "agentserver-state")) yield + + +@pytest.fixture( + params=[ + "ResponsesHostServer", + "ResponsesWorkflowHostServer", + "InvocationsHostServer", + "InvocationsWorkflowHostServer", + ] +) +def host_type( + request: pytest.FixtureRequest, +) -> type[ResponsesHostServer | ResponsesWorkflowHostServer | InvocationsHostServer | InvocationsWorkflowHostServer]: + import agent_framework_foundry_hosting + + return getattr(agent_framework_foundry_hosting, request.param) + + +@pytest.fixture +def host_constructor( + host_type: type[ + ResponsesHostServer | ResponsesWorkflowHostServer | InvocationsHostServer | InvocationsWorkflowHostServer + ], +) -> partial[Any]: + settings: dict[str, Any] + if "Workflow" in host_type.__name__: + settings = { + "workflow": lambda request: pytest.fail("Construction must not invoke the workflow factory."), + "parse_input": lambda request: pytest.fail("Construction must not invoke the parser."), + } + else: + settings = {"agent": lambda request: pytest.fail("Construction must not invoke the agent factory.")} + return partial(host_type, **settings) diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py new file mode 100644 index 00000000000..eac2b87438e --- /dev/null +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_host_options.py @@ -0,0 +1,194 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Verify explicit framework settings and the unvalidated AgentServer constructor boundary.""" + +from __future__ import annotations + +from collections.abc import Mapping +from functools import partial +from types import MappingProxyType +from typing import Any +from unittest.mock import patch + +import pytest +from azure.ai.agentserver.responses import ResponsesServerOptions + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer"], indirect=True) +def test_removed_approval_store_keyword_fails_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any] +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises(TypeError, match="function_approval_store_provider"), + ): + host_constructor(function_approval_store_provider=None) + initialize.assert_not_called() + + +@pytest.mark.parametrize( + "settings", + [ + None, + {}, + MappingProxyType({}), + {"default_model": "sdk-owned", "custom_sdk_setting": {"value": object()}}, + MappingProxyType({"steerable_conversations": True, "custom_sdk_setting": object()}), + ], +) +def test_agent_server_kwargs_are_forwarded_without_validation_or_mutation( + host_type: type[Any], host_constructor: partial[Any], settings: Mapping[str, Any] | None +) -> None: + before = dict(settings or {}) + sdk_host = host_type.__bases__[0] + with ( + patch.object(sdk_host, "__init__", side_effect=RuntimeError("SDK construction failed")) as initialize, + pytest.raises(RuntimeError, match="SDK construction failed"), + ): + host_constructor(agent_server_kwargs=settings) + initialize.assert_called_once() + forwarded = initialize.call_args.kwargs + assert "agent_server_kwargs" not in forwarded + for key, value in before.items(): + assert forwarded[key] is value + assert dict(settings or {}) == before + + +@pytest.mark.parametrize("log_level", [None, "DEBUG"]) +def test_log_level_is_forwarded_unchanged_to_agent_server( + host_type: type[Any], host_constructor: partial[Any], log_level: str | None +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__", side_effect=RuntimeError("SDK boundary")) as initialize, + pytest.raises(RuntimeError, match="SDK boundary"), + ): + host_constructor(log_level=log_level) + assert initialize.call_args.kwargs["log_level"] == log_level + + +@pytest.mark.parametrize("interval", [True, 1.5, "5", 0, -1]) +def test_invalid_explicit_heartbeats_fail_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any], interval: Any +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises((TypeError, ValueError), match="sse_keep_alive_interval_seconds"), + ): + host_constructor(sse_keep_alive_interval_seconds=interval) + initialize.assert_not_called() + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer", "ResponsesWorkflowHostServer"], indirect=True) +@pytest.mark.parametrize( + "settings", + [ + {"shutdown_grace_period_seconds": True}, + {"shutdown_grace_period_seconds": 0}, + {"shutdown_grace_period_seconds": -1}, + {"resilient_background": None}, + {"resilient_background": 1}, + ], +) +def test_invalid_responses_settings_fail_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any], settings: dict[str, Any] +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises((TypeError, ValueError), match=next(iter(settings))), + ): + host_constructor(**settings) + initialize.assert_not_called() + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer"], indirect=True) +@pytest.mark.parametrize("limit", [None, 3]) +def test_history_limit_inherits_sdk_default_or_sets_an_explicit_cap( + host_type: type[Any], host_constructor: partial[Any], monkeypatch: pytest.MonkeyPatch, limit: int | None +) -> None: + monkeypatch.setenv("DEFAULT_FETCH_HISTORY_ITEM_COUNT", "7") + with ( + patch.object(host_type.__bases__[0], "__init__", side_effect=RuntimeError("SDK boundary")) as initialize, + pytest.raises(RuntimeError, match="SDK boundary"), + ): + host_constructor(max_history_items=limit) + options = initialize.call_args.kwargs["options"] + expected = ResponsesServerOptions().default_fetch_history_count if limit is None else limit + assert options.default_fetch_history_count == expected + assert options.default_model is None + assert options.steerable_conversations is False + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer", "ResponsesWorkflowHostServer"], indirect=True) +def test_response_operational_settings_reach_the_sdk(host_type: type[Any], host_constructor: partial[Any]) -> None: + settings: dict[str, Any] = { + "sse_keep_alive_interval_seconds": 3, + "shutdown_grace_period_seconds": 2, + "resilient_background": True, + } + if host_type.__name__ == "ResponsesHostServer": + settings.update(history_source="inference_service", background_source="inference_service") + with ( + patch.object(host_type.__bases__[0], "__init__", side_effect=RuntimeError("SDK boundary")) as initialize, + pytest.raises(RuntimeError, match="SDK boundary"), + ): + host_constructor(**settings) + options = initialize.call_args.kwargs["options"] + assert options.sse_keep_alive_interval_seconds == 3 + assert options.shutdown_grace_period_seconds == 2 + assert options.resilient_background is True + + +@pytest.mark.parametrize("host_type", ["InvocationsHostServer", "InvocationsWorkflowHostServer"], indirect=True) +@pytest.mark.parametrize("environment, explicit, expected", [(None, None, 0), ("3", None, 3), ("3", 5, 5)]) +def test_invocations_heartbeat_configuration_is_per_host( + host_constructor: partial[Any], + monkeypatch: pytest.MonkeyPatch, + environment: str | None, + explicit: int | None, + expected: int, +) -> None: + if environment is None: + monkeypatch.delenv("SSE_KEEPALIVE_INTERVAL", raising=False) + else: + monkeypatch.setenv("SSE_KEEPALIVE_INTERVAL", environment) + configured = host_constructor(sse_keep_alive_interval_seconds=explicit) + inherited = host_constructor() + assert configured.config.sse_keepalive_interval == expected + assert inherited.config.sse_keepalive_interval == int(environment or "0") + assert configured.config is not inherited.config + + +@pytest.mark.parametrize("host_type", ["InvocationsHostServer"], indirect=True) +def test_legacy_agent_stream_cannot_enable_heartbeats( + host_constructor: partial[Any], monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setenv("SSE_KEEPALIVE_INTERVAL", "3") + with pytest.warns(DeprecationWarning): + server = host_constructor(legacy_wire_format=True) + assert server.config.sse_keepalive_interval == 0 + with pytest.raises(ValueError, match="framed SSE"): + host_constructor(legacy_wire_format=True, sse_keep_alive_interval_seconds=1) + + +@pytest.mark.parametrize("host_type", ["ResponsesHostServer"], indirect=True) +@pytest.mark.parametrize( + "settings", + [ + {"max_history_items": True}, + {"max_history_items": 1.5}, + {"max_history_items": 0}, + {"max_history_items": -1}, + {"max_history_items": 3, "history_source": "agent_managed"}, + {"max_history_items": 3, "history_source": "inference_service"}, + {"resilient_background": True}, + ], +) +def test_agent_setting_applicability_fails_before_sdk_construction( + host_type: type[Any], host_constructor: partial[Any], settings: dict[str, Any] +) -> None: + with ( + patch.object(host_type.__bases__[0], "__init__") as initialize, + pytest.raises((TypeError, ValueError), match=next(iter(settings))), + ): + host_constructor(**settings) + initialize.assert_not_called() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py index d49d59e2bc9..04fb312829e 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_invocations_workflow.py @@ -56,12 +56,12 @@ CheckpointStoreProvider, FoundryRequestScope, InvocationsHostServer, + InvocationsWorkflowHostServer, WorkflowSource, WorkflowTurn, ) -from agent_framework_foundry_hosting._invocations import _workflow_output, _WorkflowOutputError -from agent_framework_foundry_hosting._state_store import FoundryCheckpointStore -from agent_framework_foundry_hosting._workflow_state import FoundryWorkflowBindingStore, WorkflowHead +from agent_framework_foundry_hosting._invocations_shared import _workflow_output, _WorkflowOutputError +from agent_framework_foundry_hosting._shared import FoundryCheckpointStore, FoundryWorkflowBindingStore, WorkflowHead @dataclass(frozen=True) @@ -301,10 +301,10 @@ def _host( *, parser: Any = _parse_ticket, checkpoints: CheckpointStoreProvider | None = None, -) -> InvocationsHostServer: - return InvocationsHostServer( +) -> InvocationsWorkflowHostServer: + return InvocationsWorkflowHostServer( workflow=factory, - parse_request=parser, + parse_input=parser, checkpoint_store_provider=checkpoints or CheckpointStoreProvider( allowed_checkpoint_types=[ @@ -312,7 +312,7 @@ def _host( for value in (Ticket, TicketModel, TicketState, TicketReview, TicketDecision) ] ), - configure_observability=None, + agent_server_kwargs={"configure_observability": None}, ) @@ -339,7 +339,7 @@ async def _events(response: Response) -> list[dict[str, Any]]: async def _invoke( - server: InvocationsHostServer, + server: InvocationsWorkflowHostServer, payload: Any, *, session_id: str = "session", @@ -438,16 +438,16 @@ def create(_request: Request) -> Workflow: return WorkflowBuilder(name="tickets", start_executor=_TicketExecutor([])).build() invalid_agent: Any = object() - with pytest.raises(TypeError, match="exactly one"): - InvocationsHostServer(agent=invalid_agent, workflow=create, parse_request=_parse_ticket) - with pytest.raises(TypeError, match="parse_request is required"): - InvocationsHostServer(workflow=create) + with pytest.raises(TypeError, match="workflow"): + cast(Any, InvocationsHostServer)(agent=invalid_agent, workflow=create) + with pytest.raises(TypeError, match="parse_input"): + cast(Any, InvocationsWorkflowHostServer)(workflow=create) def test_workflow_rejects_agent_legacy_wire_format(self) -> None: - with pytest.raises(ValueError, match="legacy_wire_format.*native"): - InvocationsHostServer( + with pytest.raises(TypeError, match="legacy_wire_format"): + cast(Any, InvocationsWorkflowHostServer)( workflow=lambda _: WorkflowBuilder(name="tickets", start_executor=_TicketExecutor([])).build(), - parse_request=_parse_ticket, + parse_input=_parse_ticket, legacy_wire_format=True, ) @@ -488,27 +488,36 @@ def create(request: Request) -> Workflow: assert len(created) == 1 assert calls == [] - @pytest.mark.parametrize("stream", [False, True]) - async def test_built_workflow_allows_one_shot_output_but_rejects_pauses_without_state(self, stream: bool) -> None: + def test_built_workflow_is_rejected_before_execution_or_state_access(self) -> None: calls: list[tuple[Ticket, str | None]] = [] one_shot = WorkflowBuilder(name="one-shot", start_executor=_TicketExecutor(calls)).build() - response, events = await _invoke(_host(one_shot), {"ticket_id": "T-1", "stream": stream}) - assert response.status_code == 200 - assert [event["event"] for event in events if event["event"] == "output"] == ["output"] - assert calls == [(Ticket("T-1", "Help"), None)] - store = FoundryWorkflowBindingStore(FoundryRequestScope("session", None, None, False)) - assert await store.get_head("invocations", None) == (None, None) - - pausing = WorkflowBuilder(name="pausing", start_executor=_ReviewExecutor([])).build() - response, events = await _invoke(_host(pausing), {"ticket_id": "T-2", "stream": stream}) - assert response.status_code == (200 if stream else 400) - assert [event["event"] for event in events] == ["error"] - message = events[0].get("message") or events[0].get("error") - assert isinstance(message, str) and "request-aware factory" in message - assert await store.get_head("invocations", None) == (None, None) + with pytest.raises(TypeError, match="request-aware factory"): + _host(cast(Any, one_shot)) + assert calls == [] class TestNativeWorkflowTurns: + @pytest.mark.parametrize("stream", [False, True]) + async def test_new_wrappers_from_shared_builder_fail_before_second_dispatch(self, stream: bool) -> None: + calls: list[tuple[Ticket, str | None]] = [] + builder = WorkflowBuilder(name="shared-builder", start_executor=_TicketExecutor(calls)) + graphs: list[Workflow] = [] + + def create(request: Request) -> Workflow: + graph = builder.build() + graphs.append(graph) + return graph + + server = _host(create) + first, _ = await _invoke(server, {"ticket_id": "T-1", "stream": stream}) + second, events = await _invoke(server, {"ticket_id": "T-2", "stream": stream}) + assert first.status_code == 200 + assert second.status_code == (200 if stream else 500) + assert [event["event"] for event in events] == ["error"] + assert len(calls) == 1 + assert graphs[0] is not graphs[1] + assert graphs[0].get_start_executor() is graphs[1].get_start_executor() + async def test_checkpoint_allowlist_failure_happens_before_claim_and_dispatch(self) -> None: calls: list[tuple[Ticket, str | None]] = [] @@ -1098,18 +1107,17 @@ def factory(_request: Request) -> Workflow: assert response.status_code == 400 assert calls == resources == [] - async def test_borrowed_raw_workflow_cannot_forge_fresh_factory_authority(self) -> None: + def test_borrowed_raw_workflow_is_rejected_at_construction(self) -> None: calls: list[dict[str, Any]] = [] resources: list[tuple[str, str | None]] = [] options: ChatOptions = {"store": False, "temperature": 0.7} agent = RawAgent(client=_ContextClient(calls, resources), name="raw", default_options=options) workflow = WorkflowBuilder(name="raw-agent", start_executor=AgentExecutor(agent, id="responder")).build() - server = _host( - workflow, - parser=lambda _: WorkflowTurn(input="hello", client_kwargs={"options": {"temperature": 0.7}}), - ) - response, _ = await _invoke(server, {"fresh_factory": True}) - assert response.status_code == 400 + with pytest.raises(TypeError, match="request-aware factory"): + _host( + cast(Any, workflow), + parser=lambda _: WorkflowTurn(input="hello", client_kwargs={"options": {"temperature": 0.7}}), + ) assert calls == resources == [] @pytest.mark.parametrize("bare_raw", [False, True]) @@ -1264,11 +1272,11 @@ async def handle(self, _: Ticket, ctx: WorkflowContext[None, str]) -> None: byte_limit = 4096 if limit_kind == "events" else 128 with ( patch( - "agent_framework_foundry_hosting._invocations._MAX_WORKFLOW_SNAPSHOT_EVENTS", + "agent_framework_foundry_hosting._invocations_workflow._MAX_WORKFLOW_SNAPSHOT_EVENTS", event_limit, ), patch( - "agent_framework_foundry_hosting._invocations._MAX_WORKFLOW_SNAPSHOT_BYTES", + "agent_framework_foundry_hosting._invocations_workflow._MAX_WORKFLOW_SNAPSHOT_BYTES", byte_limit, ), ): @@ -1344,7 +1352,7 @@ async def send(message: Any) -> None: store = FoundryWorkflowBindingStore(trusted_scope) before, before_etag = await store.get_head("invocations", None) - def hosted() -> InvocationsHostServer: + def hosted() -> InvocationsWorkflowHostServer: replacement = _host(factory) replacement.config.is_hosted = True replacement.config.session_id = "" diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py index bd90420d6a0..a00942579a5 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_native_responses.py @@ -5,7 +5,6 @@ import asyncio import json import logging -import warnings from collections.abc import AsyncIterator, Awaitable, Callable, Mapping, MutableMapping, Sequence from typing import Any, cast from unittest.mock import patch @@ -36,18 +35,19 @@ ) from agent_framework._workflows._agent_utils import prepare_executor_run_kwargs from agent_framework._workflows._const import RESOLVED_WORKFLOW_RUN_KWARGS_KEY, WORKFLOW_RUN_KWARGS_KEY -from azure.ai.agentserver.responses import FileResponseStore, InMemoryResponseProvider, ResponsesServerOptions +from azure.ai.agentserver.responses import FileResponseStore, InMemoryResponseProvider from azure.ai.agentserver.responses._id_generator import IdGenerator from pydantic import BaseModel from agent_framework_foundry_hosting import ( CheckpointStoreProvider, FoundryRequestScope, - HostedResponseRequest, ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, WorkflowTurn, ) -from agent_framework_foundry_hosting._workflow_state import FoundryWorkflowBindingStore, HostedWorkflowRun +from agent_framework_foundry_hosting._shared import FoundryWorkflowBindingStore, HostedWorkflowRun class _Ticket(BaseModel): @@ -200,15 +200,15 @@ def _progress_factory( started: asyncio.Event, release: asyncio.Event, stopped: asyncio.Event, -) -> Callable[[HostedResponseRequest], Workflow]: - def create(request: HostedResponseRequest) -> Workflow: +) -> Callable[[ResponsesRequest], Workflow]: + def create(request: ResponsesRequest) -> Workflow: progress = _Progress(started, release, stopped) return WorkflowBuilder(name="progress", start_executor=progress).add_edge(progress, progress).build() return create -async def _parse_count(request: HostedResponseRequest) -> WorkflowTurn[int]: +async def _parse_count(request: ResponsesRequest) -> WorkflowTurn[int]: return WorkflowTurn(input=int(await request.get_input_text() or "")) @@ -225,14 +225,14 @@ def _build(executor: Executor | None = None, *, name: str = "native-responses") return WorkflowBuilder(name=name, start_executor=executor or _TypedCounter()).build() -async def _parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: +async def _parse(request: ResponsesRequest) -> WorkflowTurn[Any]: items = await request.get_input_items() if any(item.get("type") in ("function_call_output", "mcp_approval_response") for item in items): return WorkflowTurn(responses=await request.get_workflow_responses()) return WorkflowTurn(input=_Ticket.model_validate_json(await request.get_input_text() or "")) -async def _parse_text(request: HostedResponseRequest) -> WorkflowTurn[Any]: +async def _parse_text(request: ResponsesRequest) -> WorkflowTurn[Any]: items = await request.get_input_items() if any(item.get("type") in ("function_call_output", "mcp_approval_response") for item in items): return WorkflowTurn(responses=await request.get_workflow_responses()) @@ -245,21 +245,21 @@ def _server( parser: Callable[..., Any] = _parse, response_store: Any = None, **kwargs: Any, -) -> ResponsesHostServer: - return ResponsesHostServer( +) -> ResponsesWorkflowHostServer: + return ResponsesWorkflowHostServer( workflow=source or (lambda request: _build()), - parse_response=parser, + parse_input=parser, response_store=response_store or InMemoryResponseProvider(), checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{_Ticket.__module__}:{_Ticket.__qualname__}"], ), - configure_observability=None, + agent_server_kwargs={"configure_observability": None}, **kwargs, ) async def _post( - server: ResponsesHostServer, + server: ResponsesWorkflowHostServer | ResponsesHostServer, input: Any, *, sandbox: str = "sandbox", @@ -311,7 +311,7 @@ async def test_typed_native_turns_use_fresh_graphs_and_exact_state() -> None: graphs: list[Workflow] = [] scopes: list[FoundryRequestScope] = [] - def build(request: HostedResponseRequest) -> Workflow: + def build(request: ResponsesRequest) -> Workflow: scopes.append(request.scope) graph = _build() graphs.append(graph) @@ -345,11 +345,11 @@ def build(request: HostedResponseRequest) -> Workflow: async def test_default_checkpoint_provider_rejects_unallowlisted_typed_input_before_claim() -> None: - server = ResponsesHostServer( + server = ResponsesWorkflowHostServer( workflow=lambda request: _build(), - parse_response=_parse, + parse_input=_parse, response_store=InMemoryResponseProvider(), - configure_observability=None, + agent_server_kwargs={"configure_observability": None}, ) response = await _post(server, '{"text":"typed"}') @@ -457,8 +457,8 @@ async def test_approval_wire_ids_bind_only_to_exact_pending_checkpoint() -> None async def test_store_false_has_no_durable_native_state_and_cannot_pause() -> None: with ( - patch("agent_framework_foundry_hosting._workflow_state.FoundryStateStore.get_or_create") as bindings, - patch("agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create") as checkpoints, + patch("agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create") as bindings, + patch("agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create") as checkpoints, ): response = await _post(_server(), '{"text":"one"}', store=False) assert response["status"] == "completed", response @@ -471,21 +471,62 @@ async def test_store_false_has_no_durable_native_state_and_cannot_pause() -> Non async def test_unsupported_shared_workflow_is_not_cloned_or_unwrapped() -> None: graph = _build() - server = _server(graph) + with pytest.raises(TypeError, match="request-aware factory"): + _server(graph) + server = _server(lambda _: graph) first = await _post(server, '{"text":"one"}', store=False) assert first["status"] == "completed", first second = await _post(server, '{"text":"two"}', store=False) assert second["status"] == "failed" - with pytest.raises(TypeError, match="parse_response"): - ResponsesHostServer(workflow=_build()) - with pytest.raises(ValueError, match="exactly one"): - ResponsesHostServer() - with pytest.raises(ValueError, match="request-aware factory"): - ResponsesHostServer( - workflow=_build(), parse_response=_parse, options=ResponsesServerOptions(resilient_background=True) + with pytest.raises(TypeError, match="parse_input"): + cast(Any, ResponsesWorkflowHostServer)(workflow=lambda _: _build()) + with pytest.raises(TypeError, match="agent"): + cast(Any, ResponsesHostServer)() + with pytest.raises(TypeError, match="request-aware factory"): + ResponsesWorkflowHostServer( + workflow=cast(Any, _build()), + parse_input=_parse, + resilient_background=True, ) +async def test_new_workflow_wrapper_does_not_hide_shared_builder_executors() -> None: + builder = WorkflowBuilder(name="shared-builder", start_executor=_TypedCounter()) + graphs: list[Workflow] = [] + + def create(request: ResponsesRequest) -> Workflow: + graph = builder.build() + graphs.append(graph) + return graph + + server = _server(create) + first = await _post(server, '{"text":"one"}', store=False) + second = await _post(server, '{"text":"two"}', store=False) + assert first["status"] == "completed", first + assert second["status"] == "failed" + assert not second["output"] + assert graphs[0] is not graphs[1] + assert graphs[0].get_start_executor() is graphs[1].get_start_executor() + + +@pytest.mark.parametrize("stream", [False, True]) +async def test_workflow_host_prefix_preserves_create_and_retrieval_routes(stream: bool) -> None: + server = _server(prefix="api/v1") + async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as client: + created = await client.post("/api/v1/responses", json={"input": '{"text":"prefixed"}', "stream": stream}) + assert created.status_code == 200 + if stream: + events = [json.loads(line[6:]) for line in created.text.splitlines() if line.startswith("data: {")] + result = next(event["response"] for event in events if event["type"] == "response.completed") + else: + result = created.json() + assert result["status"] == "completed", result + assert result["output"][0]["output"] == {"text": "prefixed", "count": 1} + stored = await client.get(f"/api/v1/responses/{result['id']}") + assert stored.status_code == 200 + assert stored.json()["output"] == result["output"] + + async def test_pair_write_failure_withholds_tentative_approval_output() -> None: server = _server(lambda request: _build(_Approvals()), parser=_parse_text) original = HostedWorkflowRun.stage @@ -511,7 +552,7 @@ def __init__(self) -> None: async def start(self, message: str, ctx: WorkflowContext[str, str]) -> None: raise RuntimeError("private-provider-token at /srv/private/path") - caplog.set_level(logging.ERROR, logger="agent_framework_foundry_hosting._workflow_responses") + caplog.set_level(logging.ERROR, logger="agent_framework_foundry_hosting._responses_workflow") response = await _post(_server(lambda request: _build(_Fail()), parser=_parse_text), "fail") assert response["status"] == "failed" assert "private-provider-token" not in json.dumps(response) @@ -609,7 +650,7 @@ async def test_native_agent_executor_applies_options_without_mutating_defaults_o lifecycle: list[str] = [] agents: list[Agent[Any]] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: agent = Agent( client=_RecordingClient(calls, lifecycle), name="model", @@ -618,7 +659,7 @@ def create(request: HostedResponseRequest) -> Workflow: agents.append(agent) return _build(AgentExecutor(agent, id="model")) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", await request.get_input_text() or "")]) server = _server(create, parser=parse) @@ -641,7 +682,7 @@ async def test_registry_backed_executor_gets_native_options_and_resource_lifecyc calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: agent = Agent( client=_RecordingClient(calls, lifecycle), name="registered", @@ -663,11 +704,11 @@ async def test_request_owned_resources_close_when_pre_execution_preparation_fail calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: agent = Agent(client=_RecordingClient(calls, lifecycle), name="owned") return _build(AgentExecutor(agent, id="owned")) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: raise ValueError("malformed typed input") kwargs = {"previous_response_id": IdGenerator.new_response_id()} if failure == "continuation" else {} @@ -682,11 +723,11 @@ async def test_native_provider_that_ignores_store_false_cannot_commit_or_leak_to calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: agent = Agent(client=_RecordingClient(calls, lifecycle, violates_store=True), name="model") return _build(AgentExecutor(agent, id="model")) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", "test")]) response = await _post(_server(create, parser=parse), "one", store=False) @@ -700,10 +741,10 @@ async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: async def test_bare_raw_agent_fails_before_dispatch_instead_of_pretending_to_apply_options() -> None: calls: list[dict[str, Any]] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: return _build(AgentExecutor(RawAgent(client=_RecordingClient(calls, []), name="raw"), id="raw")) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", "test")]) response = await _post(_server(create, parser=parse), "one", temperature=0.4) @@ -717,7 +758,7 @@ async def test_raw_agent_fresh_factory_matches_explicit_defaults(stored: bool, s calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: agent = RawAgent( client=_RecordingClient(calls, lifecycle), name="raw", @@ -725,7 +766,7 @@ def create(request: HostedResponseRequest) -> Workflow: ) return _build(AgentExecutor(agent, id="raw")) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", await request.get_input_text() or "")]) server = _server(create, parser=parse) @@ -761,7 +802,7 @@ async def test_raw_agent_unverified_storage_defaults_fail_before_claim(default_s calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: options: ChatOptions[Any] = {} if default_store is not None: options["store"] = default_store @@ -776,7 +817,7 @@ def create(request: HostedResponseRequest) -> Workflow: ) ) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", "hello")]) response = await _post(_server(create, parser=parse), "hello") @@ -789,7 +830,7 @@ async def test_raw_agent_private_continuation_violation_never_publishes_output(s calls: list[dict[str, Any]] = [] lifecycle: list[str] = [] - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: return _build( AgentExecutor( RawAgent( @@ -801,7 +842,7 @@ def create(request: HostedResponseRequest) -> Workflow: ) ) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", "hello")]) response = await _post(_server(create, parser=parse), "hello", store=stored) @@ -885,7 +926,7 @@ async def __aexit__(self, *args: object) -> None: cleanup_calls.append(get_request_context().call_id) await super().__aexit__(*args) - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: agent = RawAgent( client=GatedClient([], lifecycle), name="raw", @@ -893,7 +934,7 @@ def create(request: HostedResponseRequest) -> Workflow: ) return _build(AgentExecutor(agent, id="raw")) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", "wait")]) server = _server(create, parser=parse) @@ -934,7 +975,7 @@ async def updates() -> AsyncIterator[ChatResponseUpdate]: return ResponseStream(updates(), finalizer=ChatResponse.from_updates) - def create(request: HostedResponseRequest) -> Workflow: + def create(request: ResponsesRequest) -> Workflow: return _build( AgentExecutor( RawAgent( @@ -946,7 +987,7 @@ def create(request: HostedResponseRequest) -> Workflow: ) ) - async def parse(request: HostedResponseRequest) -> WorkflowTurn[Any]: + async def parse(request: ResponsesRequest) -> WorkflowTurn[Any]: return WorkflowTurn(input=[Message("user", "wait")]) server = _server(create, parser=parse) @@ -1011,8 +1052,13 @@ async def update_response(self, response: Any, **kwargs: Any) -> None: provider = FailingCheckpointStore(storage_dir=tmp_path / "responses") factory = _progress_factory(started, release, stopped) - options = ResponsesServerOptions(resilient_background=True, shutdown_grace_period_seconds=1) - server = _server(factory, parser=_parse_count, response_store=provider, options=options) + server = _server( + factory, + parser=_parse_count, + response_store=provider, + resilient_background=True, + shutdown_grace_period_seconds=1, + ) paired = asyncio.Event() original = HostedWorkflowRun.stage @@ -1049,7 +1095,8 @@ async def stage(run: HostedWorkflowRun, snapshot: Any, **kwargs: Any) -> None: factory, parser=_parse_count, response_store=provider, - options=ResponsesServerOptions(resilient_background=True, shutdown_grace_period_seconds=1), + resilient_background=True, + shutdown_grace_period_seconds=1, ) async with ( restarted.router.lifespan_context(restarted), @@ -1062,34 +1109,3 @@ async def stage(run: HostedWorkflowRun, snapshot: Any, **kwargs: Any) -> None: finally: release.set() set_resilient_tasks_enabled(enabled) - - -async def test_legacy_wrapper_warns_once_per_host_and_keeps_message_semantics( - caplog: pytest.LogCaptureFixture, -) -> None: - class _Legacy(Executor): - def __init__(self) -> None: - super().__init__("legacy") - - @handler - async def start(self, message: list[Message], ctx: WorkflowContext[str, str]) -> None: - await ctx.yield_output(message[-1].text) - - def create() -> Any: - return _build(_Legacy()).as_agent() - - server = ResponsesHostServer(agent=create, response_store=InMemoryResponseProvider(), configure_observability=None) - with warnings.catch_warnings(record=True) as captured: - warnings.simplefilter("always", DeprecationWarning) - first = await _post(server, "legacy") - second = await _post(server, "again", previous_response_id=first["id"]) - assert first["status"] == second["status"] == "completed" - assert _texts(first) == ["legacy"] - assert _texts(second) == ["again"] - assert sum("Hosting WorkflowAgent" in str(item.message) for item in captured) == 1 - message = next(str(item.message) for item in captured if "Hosting WorkflowAgent" in str(item.message)) - assert "should be avoided" in message - assert "stateful" in message - assert "parse_response" in message - logged = [record for record in caplog.records if "DEPRECATION: Hosting WorkflowAgent" in record.getMessage()] - assert [record.levelno for record in logged] == [logging.WARNING] diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py b/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py index 30e80a15bfa..fa80156dc5a 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/test_workflow_state.py @@ -35,13 +35,14 @@ from azure.ai.agentserver.core.storage import FoundryStorageConflictError, FoundryStoragePreconditionError from agent_framework_foundry_hosting import CheckpointStoreProvider, FoundryRequestScope, WorkflowTurn -from agent_framework_foundry_hosting._workflow_source import WorkflowResolver, prepare_workflow_kwargs -from agent_framework_foundry_hosting._workflow_state import ( +from agent_framework_foundry_hosting._shared import ( FoundryWorkflowBindingStore, HostedWorkflowRun, WorkflowBlockedError, WorkflowConflictError, WorkflowHead, + WorkflowResolver, + prepare_workflow_kwargs, ) @@ -213,12 +214,41 @@ def builder_factory(request: object) -> Any: with pytest.raises(TypeError, match="built Workflow"): await builder_resolver.resolve(object()) stored_resolver: WorkflowResolver[object] = WorkflowResolver( - WorkflowBuilder(start_executor=_Counter(), checkpoint_storage=InMemoryCheckpointStorage()).build() + lambda _: WorkflowBuilder(start_executor=_Counter(), checkpoint_storage=InMemoryCheckpointStorage()).build() ) with pytest.raises(RuntimeError, match="without a store"): await stored_resolver.resolve(object()) +@pytest.mark.parametrize("wrapped_workflow", [False, True]) +@pytest.mark.parametrize("async_factory", [False, True]) +async def test_workflow_sources_reject_agents_before_dispatch(wrapped_workflow: bool, async_factory: bool) -> None: + regular_agent = Agent(client=MagicMock(spec=BaseChatClient)) + agent = ( + WorkflowBuilder(start_executor=AgentExecutor(regular_agent)).build().as_agent() + if wrapped_workflow + else regular_agent + ) + requests: list[object] = [] + + def create(request: object) -> Any: + requests.append(request) + return agent + + async def create_async(request: object) -> Any: + return create(request) + + with patch.object(agent, "run") as run: + with pytest.raises(TypeError, match="request-aware factory"): + WorkflowResolver(cast(Any, agent)) + resolver: WorkflowResolver[object] = WorkflowResolver(create_async if async_factory else create) + request = object() + with pytest.raises(TypeError, match="must return a built Workflow"): + await resolver.resolve(request) + assert requests == [request] + run.assert_not_called() + + async def test_request_aware_async_factory() -> None: requests: list[object] = [] @@ -341,14 +371,14 @@ def create(request: object) -> Workflow: resolver = WorkflowResolver(create) workflow = await resolver.resolve(object()) - run = await _prepare("model-turn", workflow=workflow, fresh_factory=resolver.is_factory) + run = await _prepare("model-turn", workflow=workflow, fresh_factory=True) turn = run.validate_turn(WorkflowTurn(input=[Message("user", "hello")])) clients, functions = prepare_workflow_kwargs( workflow, turn, run.scope, options={"temperature": 0.6}, - fresh_factory=resolver.is_factory, + fresh_factory=True, ) await run.claim() events = [event async for event in run.events(turn, client_kwargs=clients, function_invocation_kwargs=functions)] @@ -422,7 +452,7 @@ async def test_completed_response_replay_is_explicit_and_does_not_advance_the_he async def test_default_store_integrity_hashes_the_persisted_encoding_without_repickling() -> None: first = await _prepare("one", workflow=_workflow(_Approval())) with patch( - "agent_framework_foundry_hosting._workflow_state._checkpoint_hash", + "agent_framework_foundry_hosting._shared._checkpoint_hash", side_effect=AssertionError("default storage must not re-pickle checkpoints for integrity"), ): await _finish(first, WorkflowTurn(input="/safe")) @@ -542,13 +572,13 @@ async def test_both_sdk_cas_errors_are_visible_and_private(error: type[Exception async def test_store_false_writes_nothing_and_rejects_resumption() -> None: - with patch("agent_framework_foundry_hosting._workflow_state.FoundryStateStore.get_or_create") as get_store: + with patch("agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create") as get_store: run = await _prepare("one", stored=False) assert await _finish(run, WorkflowTurn(input=3)) == [3] get_store.assert_not_called() with pytest.raises(ValueError, match="one-shot"): await _prepare("two", stored=False, previous_response_id="one") - with patch("agent_framework_foundry_hosting._workflow_state.FoundryStateStore.get_or_create") as get_store: + with patch("agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create") as get_store: run = await _prepare("pause", workflow=_workflow(_Review()), stored=False) turn = run.validate_turn(WorkflowTurn(input="review")) await run.claim() diff --git a/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py b/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py index 81fd0550e76..6990c67df2b 100644 --- a/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py +++ b/python/packages/foundry_hosting/tests/foundry_hosting/toolbox_approval_helpers.py @@ -1,27 +1,44 @@ # Copyright (c) Microsoft. All rights reserved. -"""Offline protocol boundaries for Toolbox approval regressions.""" +"""Offline protocol boundaries for Toolbox approval and OAuth consent regressions.""" from __future__ import annotations +import asyncio +import copy import json -from collections.abc import AsyncGenerator, AsyncIterator, Awaitable, Mapping, Sequence +from collections.abc import AsyncGenerator, AsyncIterator, Awaitable, Callable, Mapping, Sequence from contextlib import asynccontextmanager -from typing import Any +from importlib import import_module +from typing import Any, Literal +import httpx as host_httpx import httpx2 as httpx from agent_framework import ( BaseChatClient, ChatResponse, ChatResponseUpdate, Content, + FunctionInvocationContext, FunctionInvocationLayer, + FunctionMiddleware, + FunctionTool, Message, ResponseStream, + SessionStore, ) +from agent_framework.exceptions import UserInputRequiredException +from agent_framework.openai import OpenAIChatClient, OpenAIChatOptions +from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext from azure.core.credentials import AccessToken +from openai import AsyncOpenAI, DefaultAsyncHttpxClient +from pydantic import BaseModel -from agent_framework_foundry_hosting import FoundryToolbox +from agent_framework_foundry_hosting import FoundryToolbox, ResponsesHostServer +from agent_framework_foundry_hosting._shared import StoreProvider + +TOOL_NAMES = ("learn___completed", "learn___protected", "learn___later") +CONSENT_LINK = "https://consent.example.com/authorize" class ToolboxCredential: @@ -150,3 +167,300 @@ async def complete() -> ChatResponse: return ChatResponse(messages=[Message("assistant", contents=contents)]) return ResponseStream(updates(), finalizer=ChatResponse.from_updates) if stream else complete() + + +class ToolboxConsentServer(ToolboxApprovalServer): + def __init__(self) -> None: + super().__init__() + self.tools = [toolbox_tool("never", name=name) for name in TOOL_NAMES] + self.authorized = False + self.before_consent: Callable[[], Awaitable[object]] | None = None + self.error: dict[str, Any] = { + "code": -32006, + "message": "tools/call failed " + + json.dumps({ + "errors": [ + { + "name": "learn", + "type": "mcp", + "error": {"code": "CONSENT_REQUIRED", "message": CONSENT_LINK}, + } + ] + }), + } + + async def handle(self, request: httpx.Request) -> httpx.Response: + if request.method == "POST": + payload = json.loads(request.content) + if ( + payload.get("method") == "tools/call" + and payload["params"]["name"] == "learn___protected" + and not self.authorized + ): + self.calls.append(payload["params"]) + if self.before_consent is not None: + await self.before_consent() + return httpx.Response( + 200, + json={"jsonrpc": "2.0", "id": payload["id"], "error": self.error}, + ) + return await super().handle(request) + + +class ConsentModelServer: + def __init__(self, tool_names: Sequence[str] = TOOL_NAMES) -> None: + self.tool_names = tuple(tool_names) + self.requests: list[dict[str, Any]] = [] + self.chains: dict[str, list[dict[str, Any]]] = {} + self.history_errors: list[str] = [] + self.fail_settlement = False + self.unexpected_settlement_call = False + self.sdk_httpx: Any = import_module(DefaultAsyncHttpxClient.__mro__[1].__module__.partition(".")[0]) + + async def handle(self, request: Any) -> Any: + payload = json.loads(request.content) + self.requests.append(payload) + previous = payload.get("previous_response_id") + history = copy.deepcopy(self.chains[previous]) if previous is not None else [] + inputs = payload["input"] + history.extend(inputs if isinstance(inputs, list) else [{"type": "message", "content": inputs}]) + pending: set[str] = set() + for item in history: + if item.get("type") == "function_call": + pending.add(item["call_id"]) + elif item.get("type") == "function_call_output": + if item["call_id"] not in pending: + self.history_errors.append(f"Unexpected tool result for {item['call_id']}") + pending.discard(item["call_id"]) + if pending or self.history_errors: + self.history_errors.extend(f"No tool output found for {call_id}" for call_id in sorted(pending)) + return self.sdk_httpx.Response( + 400, + json={"error": {"message": self.history_errors[-1], "type": "invalid_request_error"}}, + ) + settling = payload.get("tool_choice") == "none" + if settling and self.fail_settlement: + return self.sdk_httpx.Response( + 500, json={"error": {"message": "settlement refused", "type": "server_error"}} + ) + index = len(self.requests) + names = self.tool_names if index == 1 else () + authorized_retry = any(item.get("role") == "user" and "I have signed in" in json.dumps(item) for item in inputs) + if ( + not settling + and authorized_retry + and not any( + item.get("type") == "function_call_output" and item["call_id"].startswith("call_retry_") + for item in inputs + ) + ): + names = ("learn___protected",) + if settling and self.unexpected_settlement_call: + names = ("learn___later",) + output: list[dict[str, Any]] = [ + { + "type": "function_call", + "id": f"fc_{index}_{position}", + "call_id": f"call_{'initial' if index == 1 else 'retry'}_{position}", + "name": name, + "arguments": "{}", + "status": "completed", + } + for position, name in enumerate(names) + ] + if not output: + output = [ + { + "type": "message", + "id": f"msg_{index}", + "role": "assistant", + "status": "completed", + "content": [ + { + "type": "output_text", + "text": "hidden settlement reply" if settling else "done", + "annotations": [], + "logprobs": [], + } + ], + } + ] + response: dict[str, Any] = { + "id": f"resp_model_{index}", + "object": "response", + "created_at": 1_700_000_000, + "model": "test-model", + "status": "completed", + "output": output, + "error": None, + "incomplete_details": None, + "instructions": None, + "metadata": {}, + "parallel_tool_calls": True, + "temperature": 1.0, + "tool_choice": payload.get("tool_choice", "auto"), + "tools": [], + "top_p": 1.0, + "usage": { + "input_tokens": 5, + "output_tokens": 2, + "total_tokens": 7, + "input_tokens_details": {"cached_tokens": 0}, + "output_tokens_details": {"reasoning_tokens": 0}, + }, + "store": payload.get("store", True), + "background": False, + "conversation": None, + } + if response["store"]: + self.chains[response["id"]] = history + copy.deepcopy(output) + if not payload.get("stream"): + return self.sdk_httpx.Response(200, json=response) + events: list[dict[str, Any]] = [ + { + "type": "response.created", + "response": {**response, "status": "in_progress", "output": [], "usage": None}, + } + ] + for position, item in enumerate(output): + events.append({ + "type": "response.output_item.added", + "output_index": position, + "item": {**item, "arguments": ""} if item["type"] == "function_call" else {**item, "content": []}, + }) + if item["type"] == "function_call": + events.append({ + "type": "response.function_call_arguments.delta", + "item_id": item["id"], + "output_index": position, + "delta": item["arguments"], + }) + else: + events.append({ + "type": "response.output_text.delta", + "item_id": item["id"], + "output_index": position, + "content_index": 0, + "delta": item["content"][0]["text"], + "logprobs": [], + }) + events.append({"type": "response.output_item.done", "output_index": position, "item": item}) + events.append({"type": "response.completed", "response": response}) + for sequence_number, event in enumerate(events): + event["sequence_number"] = sequence_number + text = "".join(f"event: {event['type']}\ndata: {json.dumps(event)}\n\n" for event in events) + return self.sdk_httpx.Response( + 200, headers={"content-type": "text/event-stream"}, text=text + "data: [DONE]\n\n" + ) + + +@asynccontextmanager +async def consent_model_client( + server: ConsentModelServer, *, concurrent: bool = False +) -> AsyncGenerator[OpenAIChatClient[OpenAIChatOptions]]: + async with AsyncOpenAI( + api_key="offline-key", + base_url="https://model.invalid/v1", + max_retries=0, + http_client=DefaultAsyncHttpxClient(transport=server.sdk_httpx.MockTransport(server.handle)), + ) as api_client: + yield OpenAIChatClient[OpenAIChatOptions]( + model="test-model", + async_client=api_client, + function_invocation_configuration={"allow_concurrent_invocation": concurrent}, + ) + + +class ConsentSessionStoreProvider(StoreProvider[SessionStore]): + def __init__(self) -> None: + self.store = SessionStore() + + def get_store(self, *, config: AgentConfig, platform_context: FoundryAgentRequestContext) -> SessionStore: + return self.store + + +class ConsentPendingInput(FunctionMiddleware): + def __init__(self, kind: Literal["input", "approval"], *, require_approval: bool = False) -> None: + self.kind = kind + self.executions = 0 + self.function = FunctionTool( + name="awaiting_input", + func=self.invoke, + approval_mode="always_require" if require_approval else "never_require", + ) + + async def invoke(self) -> str: + self.executions += 1 + if self.kind == "input": + raise UserInputRequiredException([ + Content("text", text="Please supply the missing input.", user_input_request=True), + Content("text", text="Return the input as this call's result.", user_input_request=True), + ]) + return "pending-result" + + async def process(self, context: FunctionInvocationContext, call_next: Callable[[], Awaitable[None]]) -> None: + if context.function.name == self.function.name and self.kind == "approval": + approval = context.metadata.get("approval_response") + approved_call = approval.function_call if isinstance(approval, Content) else None + if approved_call is None or not approved_call.additional_properties.get("test_runtime_approval"): + occurrence_id = context.metadata["function_call_occurrence_id"] + context.result = Content.from_function_approval_request( + id=f"runtime-{occurrence_id}", + function_call=Content.from_function_call( + call_id=context.metadata["call_id"], + name=context.function.name, + arguments=( + context.arguments.model_dump() + if isinstance(context.arguments, BaseModel) + else context.arguments + ), + id=occurrence_id, + additional_properties={"test_runtime_approval": True}, + ), + additional_properties={"_replacement_approval_request": True}, + ) + return + await call_next() + + +async def post_consent_response( + host: ResponsesHostServer, payload: dict[str, Any] +) -> tuple[dict[str, Any], list[dict[str, Any]]]: + async with host_httpx.AsyncClient(transport=host_httpx.ASGITransport(app=host), base_url="http://test") as client: + response = await client.post("/responses", json={"model": "test-model", **payload}) + if payload.get("background"): + assert response.status_code == 200, response.text + body = response.json() + async with asyncio.timeout(5): + while body["status"] in ("queued", "in_progress"): + await asyncio.sleep(0.01) + response = await client.get(f"/responses/{body['id']}") + assert response.status_code == 200, response.text + body = response.json() + return body, [] + assert response.status_code == 200, response.text + if not payload.get("stream"): + return response.json(), [] + events = [json.loads(line[6:]) for line in response.text.splitlines() if line.startswith("data: {")] + terminal = [ + event["response"] + for event in events + if event["type"] in ("response.completed", "response.incomplete", "response.failed") + ] + assert len(terminal) == 1, events + return terminal[0], events + + +class ConsentCallerHeaders(FunctionMiddleware): + def __init__(self) -> None: + self.later_queued = asyncio.Event() + + async def process(self, context: FunctionInvocationContext, call_next: Callable[[], Awaitable[None]]) -> None: + context.kwargs["caller"] = "later" if context.function.name.endswith("later") else "first" + await call_next() + + def headers(self, kwargs: dict[str, Any]) -> dict[str, str]: + caller = kwargs["caller"] + if caller == "later": + self.later_queued.set() + return {"x-test-caller": caller} diff --git a/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py b/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py index cf7b987f6f6..a028763bb99 100644 --- a/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py +++ b/python/packages/foundry_hosting/tests/test_foundry_hosting_imports.py @@ -1,8 +1,17 @@ # Copyright (c) Microsoft. All rights reserved. +from __future__ import annotations + +import inspect import subprocess import sys import textwrap +from typing import Any, cast +from unittest.mock import patch + +import pytest + +import agent_framework_foundry_hosting as hosting def test_package_import_is_lazy() -> None: @@ -12,23 +21,206 @@ def test_package_import_is_lazy() -> None: import agent_framework_foundry_hosting - assert "agent_framework_foundry_hosting._invocations" not in sys.modules - assert "agent_framework_foundry_hosting._responses" not in sys.modules + assert "agent_framework_foundry_hosting._invocations_agent" not in sys.modules + assert "agent_framework_foundry_hosting._invocations_shared" not in sys.modules + assert "agent_framework_foundry_hosting._invocations_workflow" not in sys.modules + assert "agent_framework_foundry_hosting._responses_agent" not in sys.modules + assert "agent_framework_foundry_hosting._responses_shared" not in sys.modules + assert "agent_framework_foundry_hosting._responses_workflow" not in sys.modules assert "agent_framework_foundry_hosting._scope" not in sys.modules - assert "agent_framework_foundry_hosting._state_store" not in sys.modules assert "agent_framework_foundry_hosting._toolbox" not in sys.modules - assert "agent_framework_foundry_hosting._workflow_source" not in sys.modules - assert "agent_framework_foundry_hosting._workflow_state" not in sys.modules + assert "agent_framework_foundry_hosting._shared" not in sys.modules """ ) subprocess.run([sys.executable, "-I", "-c", code], check=True) -def test_native_workflow_exports_match_runtime_and_typing_namespace() -> None: +@pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) +@pytest.mark.parametrize("workflow_first", [False, True]) +def test_protocol_hosts_import_independently_in_either_order(protocol: str, workflow_first: bool) -> None: + agent = f"{protocol}HostServer" + workflow = f"{protocol}WorkflowHostServer" + names = (workflow, agent) if workflow_first else (agent, workflow) + other_module = "agent" if workflow_first else "workflow" + base = "ResponsesAgentServerHost" if protocol == "Responses" else "InvocationAgentServerHost" + sdk_module = "responses.hosting" if protocol == "Responses" else "invocations" + code = textwrap.dedent( + f""" + import sys + + from agent_framework_foundry_hosting import {names[0]} + assert "agent_framework_foundry_hosting._{protocol.lower()}_{other_module}" not in sys.modules + from agent_framework_foundry_hosting import {names[1]} + from azure.ai.agentserver.{sdk_module} import {base} + + assert {agent}.__bases__ == ({base},) + assert {workflow}.__bases__ == ({base},) + """ + ) + subprocess.run([sys.executable, "-I", "-c", code], check=True) + + +def test_native_workflow_and_storage_exports_match_runtime_and_typing_namespace() -> None: import agent_framework.foundry as foundry import agent_framework_foundry_hosting as hosting - for name in ("WorkflowTurn", "WorkflowSource", "response_input_messages"): + for name in ( + "AgentSessionStoreProvider", + "CheckpointStoreProvider", + "ContextScopedStoreProvider", + "FoundryAgentSessionStore", + "FoundryCheckpointStore", + "StoreProvider", + "WorkflowTurn", + "WorkflowSource", + "ResponsesRequest", + "response_input_messages", + "ResponsesWorkflowHostServer", + "InvocationsWorkflowHostServer", + ): assert name in hosting.__all__ + assert name in dir(hosting) + assert name in dir(foundry) assert getattr(foundry, name) is getattr(hosting, name) + + +@pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) +def test_host_signatures_expose_only_the_selected_execution_model(protocol: str) -> None: + agent_host = getattr(hosting, f"{protocol}HostServer") + workflow_host = getattr(hosting, f"{protocol}WorkflowHostServer") + agent_parameters = inspect.signature(agent_host).parameters + workflow_parameters = inspect.signature(workflow_host).parameters + parser = "parse_input" + + assert agent_parameters["agent"].default is inspect.Parameter.empty + assert "workflow" not in agent_parameters + assert workflow_parameters["workflow"].default is inspect.Parameter.empty + assert workflow_parameters[parser].default is inspect.Parameter.empty + assert workflow_parameters[parser].kind is inspect.Parameter.KEYWORD_ONLY + for legacy in ("parse_response", "parse_request"): + assert legacy not in agent_parameters + assert legacy not in workflow_parameters + assert "checkpoint_store_provider" in workflow_parameters + assert "checkpoint_store_provider" not in agent_parameters + for name in ("agent", "agent_session_store_provider", "history_source", "background_source", "unsupported_options"): + assert name not in workflow_parameters + assert ("prepare_options" in workflow_parameters) is (protocol == "Responses") + for parameters in (agent_parameters, workflow_parameters): + for name in ("log_level", "agent_server_kwargs"): + assert parameters[name].default is None + assert parameters[name].kind is inspect.Parameter.KEYWORD_ONLY + assert all(parameter.kind is not inspect.Parameter.VAR_KEYWORD for parameter in parameters.values()) + if protocol == "Responses": + assert "response_store" in agent_parameters + assert "store" not in agent_parameters + + +@pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) +@pytest.mark.parametrize( + "factory", + [ + pytest.param(lambda: None, id="zero-argument"), + pytest.param(lambda model="gpt-4": None, id="optional-configuration"), + pytest.param(lambda request=None: None, id="optional-request"), + pytest.param(lambda _request: None, id="different-parameter-name"), + pytest.param(lambda request, /: None, id="positional-only-request"), + pytest.param(lambda *request: None, id="variadic-positional-request"), + pytest.param(lambda **request: None, id="variadic-keyword-request"), + pytest.param(lambda request, dependency: None, id="required-positional-dependency"), + pytest.param(lambda request, *, dependency: None, id="required-keyword-dependency"), + pytest.param(dict, id="uninspectable"), + ], +) +def test_agent_hosts_reject_ambiguous_factories_before_sdk_construction(protocol: str, factory: Any) -> None: + host = getattr(hosting, f"{protocol}HostServer") + base = ( + "azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost" + if protocol == "Responses" + else "azure.ai.agentserver.invocations.InvocationAgentServerHost" + ) + with patch(f"{base}.__init__", return_value=None) as initialize: + with pytest.raises(TypeError, match="required named 'request' parameter"): + host(agent=factory) + initialize.assert_not_called() + + +@pytest.mark.parametrize( + ("protocol", "setting"), + [ + ("Responses", "agent"), + ("Responses", "agent_session_store_provider"), + ("Responses", "function_approval_store_provider"), + ("Responses", "history_source"), + ("Responses", "background_source"), + ("Responses", "unsupported_options"), + ("Responses", "store"), + ("Invocations", "agent"), + ("Invocations", "agent_session_store_provider"), + ("Invocations", "prepare_options"), + ("Invocations", "unsupported_options"), + ("Invocations", "legacy_wire_format"), + ], +) +def test_workflow_hosts_reject_agent_settings_before_sdk_construction(protocol: str, setting: str) -> None: + workflow_host = getattr(hosting, f"{protocol}WorkflowHostServer") + parser = "parse_input" + values: dict[str, Any] = {parser: lambda _: None, setting: None} + base = ( + "azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost" + if protocol == "Responses" + else "azure.ai.agentserver.invocations.InvocationAgentServerHost" + ) + with patch(f"{base}.__init__", return_value=None) as initialize: + with pytest.raises(TypeError, match=setting): + workflow_host(workflow=lambda _: None, **values) + initialize.assert_not_called() + + +@pytest.mark.parametrize("protocol", ["Responses", "Invocations"]) +def test_workflow_hosts_validate_factories_and_parsers_at_construction(protocol: str) -> None: + workflow_host = getattr(hosting, f"{protocol}WorkflowHostServer") + parser = "parse_input" + with pytest.raises(TypeError, match=parser): + workflow_host(workflow=lambda _: None, **{parser: None}) + with pytest.raises(TypeError, match="one request"): + workflow_host(workflow=lambda: None, **{parser: lambda _: None}) + + +def test_legacy_responses_request_name_is_not_exported() -> None: + assert "HostedResponseRequest" not in hosting.__all__ + assert "HostedResponseRequest" not in dir(hosting) + with pytest.raises(AttributeError, match="HostedResponseRequest"): + hosting.__getattr__("HostedResponseRequest") + + +@pytest.mark.parametrize( + ("name", "legacy"), + [ + ("ResponsesWorkflowHostServer", "parse_response"), + ("InvocationsHostServer", "parse_request"), + ("InvocationsWorkflowHostServer", "parse_request"), + ], +) +def test_legacy_parser_names_are_not_silently_forwarded_to_sdk(name: str, legacy: str) -> None: + host = getattr(hosting, name) + source = "workflow" if "Workflow" in name else "agent" + values = {source: lambda _: None, "parse_input": lambda _: None, legacy: lambda _: None} + base = ( + "azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost" + if name.startswith("Responses") + else "azure.ai.agentserver.invocations.InvocationAgentServerHost" + ) + with patch(f"{base}.__init__", return_value=None) as initialize: + with pytest.raises(TypeError, match=legacy): + host(**values) + initialize.assert_not_called() + + +def test_responses_workflow_host_rejects_removed_options() -> None: + with pytest.raises(TypeError, match="options"): + cast(Any, hosting.ResponsesWorkflowHostServer)( + workflow=lambda _: None, + parse_input=lambda _: None, + options=None, + ) diff --git a/python/packages/foundry_hosting/tests/test_invocations.py b/python/packages/foundry_hosting/tests/test_invocations.py index 806a2283a72..e1660ece85d 100644 --- a/python/packages/foundry_hosting/tests/test_invocations.py +++ b/python/packages/foundry_hosting/tests/test_invocations.py @@ -3,7 +3,7 @@ """Unit tests for InvocationsHostServer. These tests exercise ``InvocationsHostServer`` directly by constructing the -host, driving ``_partition_key`` and ``_handle_invoke`` with a fake agent and +host, driving the shared partition-key helper and ``_handle_invoke`` with a fake agent and mock requests. The Foundry request context is injected via the public ``set_request_context`` / ``reset_request_context`` helpers rather than by patching, matching the style used in ``test_toolbox.py``. @@ -15,12 +15,10 @@ import json import logging import uuid -import warnings import weakref from collections.abc import AsyncIterator, Awaitable, Callable, Generator, Mapping, Sequence from contextlib import contextmanager from itertools import product -from pathlib import Path from typing import Any, Literal, cast from unittest.mock import AsyncMock, MagicMock, patch @@ -62,7 +60,8 @@ from starlette.responses import Response, StreamingResponse from agent_framework_foundry_hosting import InvocationRun, InvocationsHostServer, StoreProvider -from agent_framework_foundry_hosting._state_store import FoundryAgentSessionStore +from agent_framework_foundry_hosting._invocations_shared import invocation_partition_key +from agent_framework_foundry_hosting._shared import FoundryAgentSessionStore pytestmark = pytest.mark.filterwarnings("ignore:.*SessionStore is experimental.*") @@ -307,18 +306,6 @@ def _make_workflow_agent() -> WorkflowAgent: return WorkflowBuilder(name="legacy", start_executor=_EchoExecutor()).build().as_agent() -def _workflow_agent_warnings(captured: Sequence[warnings.WarningMessage]) -> list[warnings.WarningMessage]: - return [item for item in captured if "Hosting WorkflowAgent" in str(item.message)] - - -def _workflow_agent_logs(caplog: pytest.LogCaptureFixture) -> list[logging.LogRecord]: - return [ - record - for record in caplog.records - if record.levelno == logging.WARNING and "DEPRECATION: Hosting WorkflowAgent" in record.getMessage() - ] - - async def _invoke_text(server: InvocationsHostServer, message: str) -> str: request = _make_request({"message": message}) with _request_context(session_id="session"): @@ -380,7 +367,7 @@ def update_session(session: AgentSession) -> None: identities = [("sandbox-a", "user-1"), ("sandbox-b", "user-1"), ("sandbox-a", "user-2")] with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=capture_store_name, ): for round_number in (1, 2): @@ -658,24 +645,28 @@ def test_accepts_supports_agent_run(self) -> None: @pytest.mark.parametrize("agent", [None, 42]) def test_rejects_invalid_agent_source(self, agent: Any) -> None: - with pytest.raises(TypeError, match="agent must be an agent instance or a zero-argument callable"): + with pytest.raises(TypeError, match="agent must be an agent instance or a callable"): InvocationsHostServer(agent) - def test_rejects_agent_class_requiring_constructor_arguments(self) -> None: - with pytest.raises(TypeError, match="agent callable must accept no arguments"): - InvocationsHostServer(cast(Any, _ContextAgent)) + def test_rejects_agent_class_requiring_multiple_constructor_arguments(self) -> None: + class InvalidAgent: + def __init__(self, request: Request, dependency: object) -> None: + pass + + with pytest.raises(TypeError, match="agent factory must declare a required named 'request' parameter"): + InvocationsHostServer(cast(Any, InvalidAgent)) - def test_rejects_factory_requiring_arguments(self) -> None: - def create_agent(name: str) -> _FakeAgent: + def test_rejects_factory_requiring_multiple_arguments(self) -> None: + def create_agent(request: Request, name: str) -> _FakeAgent: return _make_agent(response_text=name) - with pytest.raises(TypeError, match="agent callable must accept no arguments"): + with pytest.raises(TypeError, match="agent factory must declare a required named 'request' parameter"): InvocationsHostServer(cast(Any, create_agent)) @pytest.mark.parametrize( ("parameter", "value", "error"), [ - ("parse_request", 42, "parse_request must be a callable"), + ("parse_input", 42, "parse_input must be a callable"), ("prepare_options", 42, "prepare_options must be a callable"), ("unsupported_options", "discard", "unsupported_options"), ("legacy_wire_format", "false", "legacy_wire_format must be a boolean"), @@ -686,49 +677,46 @@ def test_rejects_invalid_configuration(self, parameter: str, value: Any, error: InvocationsHostServer(_make_agent(response_text="ok"), **{parameter: value}) -class TestLegacyWorkflowAgentHosting: - async def test_instance_warns_once_per_host(self, caplog: pytest.LogCaptureFixture) -> None: - with warnings.catch_warnings(record=True) as captured: - warnings.simplefilter("always", DeprecationWarning) - server = InvocationsHostServer(_make_workflow_agent()) - texts = [await _invoke_text(server, "one"), await _invoke_text(server, "two")] +# endregion - deprecations = _workflow_agent_warnings(captured) - assert len(deprecations) == 1 - assert deprecations[0].category is DeprecationWarning - message = str(deprecations[0].message) - assert "should be avoided" in message - assert "stateful" in message - assert "parse_request" in message - assert "stateless, single-turn" in message - assert Path(deprecations[0].filename).name == Path(__file__).name - assert len(_workflow_agent_logs(caplog)) == 1 - assert texts == ["one", "two"] - - async def test_factory_warns_once_on_first_request(self, caplog: pytest.LogCaptureFixture) -> None: - with warnings.catch_warnings(record=True) as captured: - warnings.simplefilter("always", DeprecationWarning) - server = InvocationsHostServer(_make_workflow_agent) - assert _workflow_agent_warnings(captured) == [] - assert _workflow_agent_logs(caplog) == [] - texts = [await _invoke_text(server, "one"), await _invoke_text(server, "two")] - - assert len(_workflow_agent_warnings(captured)) == 1 - assert len(_workflow_agent_logs(caplog)) == 1 - assert texts == ["one", "two"] - - async def test_regular_agent_does_not_warn(self, caplog: pytest.LogCaptureFixture) -> None: - with warnings.catch_warnings(record=True) as captured: - warnings.simplefilter("always", DeprecationWarning) - server = InvocationsHostServer(_make_agent(response_text="hi")) - text = await _invoke_text(server, "hello") - - assert text == "hi" - assert _workflow_agent_warnings(captured) == [] - assert _workflow_agent_logs(caplog) == [] +class TestWorkflowAgentRejection: + def test_rejects_instance_before_sdk_construction(self) -> None: + agent = _make_workflow_agent() + with ( + patch("azure.ai.agentserver.invocations.InvocationAgentServerHost.__init__") as initialize, + pytest.raises(TypeError, match="do not support WorkflowAgent"), + ): + InvocationsHostServer(agent) + initialize.assert_not_called() -# endregion + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("async_factory", [False, True]) + async def test_rejects_factory_result_before_dispatch(self, stream: bool, async_factory: bool) -> None: + agent = _make_workflow_agent() + + def create(*, request: Request) -> WorkflowAgent: + return agent + + async def create_async(*, request: Request) -> WorkflowAgent: + return agent + + server = InvocationsHostServer(create_async if async_factory else create) + with ( + patch.object(agent, "run") as run, + _request_context(session_id="session"), + ): + response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] + _make_request({"message": "hello", "stream": stream}) + ) + if stream: + assert isinstance(response, StreamingResponse) + body = await _collect_stream(response) + assert "error" in body + assert "done" not in body + else: + assert response.status_code == 500 + run.assert_not_called() class TestParsedRequests: @@ -764,7 +752,7 @@ async def prepare(incoming: Request, options: dict[str, Any]) -> dict[str, Any]: options.pop("store") return {"temperature": options["temperature"], "reasoning_effort": options["reasoning_effort"]} - server = InvocationsHostServer(agent, parse_request=parse, prepare_options=prepare) + server = InvocationsHostServer(agent, parse_input=parse, prepare_options=prepare) with _request_context(session_id="parsed"): response = await server._handle_invoke(request) # pyright: ignore[reportPrivateUsage] assert await _success_text(response) == "ok" @@ -809,18 +797,18 @@ async def test_default_parser_rejects_invalid_payload_without_using_storage(self @pytest.mark.parametrize("result", [None, "hello", {"messages": "hello"}]) async def test_custom_parser_must_return_invocation_run(self, result: Any) -> None: agent = _make_agent(response_text="ok") - server = InvocationsHostServer(agent, parse_request=lambda _request: result) + server = InvocationsHostServer(agent, parse_input=lambda _request: result) with _request_context(session_id="invalid"): response = await server._handle_invoke(_make_request({"message": "hi"})) # pyright: ignore[reportPrivateUsage] assert response.status_code == 400 - assert json.loads(bytes(response.body)) == {"error": "parse_request must return InvocationRun."} + assert json.loads(bytes(response.body)) == {"error": "parse_input must return InvocationRun."} assert agent.calls == [] async def test_parser_failure_is_logged_but_not_exposed(self, caplog: pytest.LogCaptureFixture) -> None: def broken_parser(_request: Request) -> InvocationRun: raise RuntimeError("internal parser secret") - server = InvocationsHostServer(_make_agent(response_text="ok"), parse_request=broken_parser) + server = InvocationsHostServer(_make_agent(response_text="ok"), parse_input=broken_parser) with _request_context(session_id="invalid"): response = await server._handle_invoke(_make_request({"message": "hi"})) # pyright: ignore[reportPrivateUsage] assert response.status_code == 500 @@ -874,14 +862,14 @@ async def test_agent_control_options_are_rejected_before_storage_or_agent_creati provider = _SessionStoreProvider(_mock_session_store()) options = {field: value} - def create_agent() -> _FakeAgent: + def create_agent(*, request: Request) -> _FakeAgent: factory_calls.append("created") return agent server = InvocationsHostServer( create_agent, agent_session_store_provider=provider, - parse_request=(lambda _request: InvocationRun(messages="hi", options=options, stream=stream)) + parse_input=(lambda _request: InvocationRun(messages="hi", options=options, stream=stream)) if source == "parser" else None, prepare_options=(lambda _request, _options: options) if source == "hook" else None, @@ -977,7 +965,7 @@ async def test_unsupported_options_policy( async def test_factory_options_failure_is_framed_before_first_delta(self) -> None: provider = _SessionStoreProvider(_mock_session_store()) server = InvocationsHostServer( - lambda: _NoOptionsAgent(), + lambda request: _NoOptionsAgent(), agent_session_store_provider=provider, unsupported_options="error", ) @@ -1129,7 +1117,7 @@ async def test_http_route_restores_session_across_hosts_and_factory_agents(self) def increment(session: AgentSession) -> None: session.state["turn"] = session.state.get("turn", 0) + 1 - def create_agent() -> _FakeAgent: + def create_agent(*, request: Request) -> _FakeAgent: agent = _make_agent(response_text="ok", stream_texts=["ok"], update_session=increment) agents.append(agent) return agent @@ -1244,7 +1232,7 @@ async def updates() -> AsyncIterator[AgentResponseUpdate]: assert persisted.state == {"partial": True} async def test_factory_failure_before_first_sse_event_is_sanitized(self, caplog: pytest.LogCaptureFixture) -> None: - async def unavailable_agent() -> _FakeAgent: + async def unavailable_agent(*, request: Request) -> _FakeAgent: raise RuntimeError("secret agent initialization failure") provider = _SessionStoreProvider(_mock_session_store()) @@ -1261,7 +1249,7 @@ async def unavailable_agent() -> _FakeAgent: assert provider.contexts == [] async def test_http_stream_failure_before_first_delta_is_an_error_event(self) -> None: - async def unavailable_agent() -> _FakeAgent: + async def unavailable_agent(*, request: Request) -> _FakeAgent: raise RuntimeError("private factory details") server = InvocationsHostServer(unavailable_agent) @@ -1438,17 +1426,17 @@ class TestPartitionKey: def test_local_returns_session_id(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) with _request_context(session_id="sess-1"): - assert server._partition_key() == "sess-1" # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == "sess-1" def test_local_missing_session_id_raises(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) with _request_context(), pytest.raises(RuntimeError, match="missing session_id"): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) def test_local_ignores_user_id(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) with _request_context(session_id="sess-1", user_id="user-1"): - assert server._partition_key() == "sess-1" # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == "sess-1" @pytest.mark.parametrize("user_id", [None, ""]) def test_hosted_requires_user_id(self, user_id: str | None) -> None: @@ -1459,21 +1447,21 @@ def test_hosted_requires_user_id(self, user_id: str | None) -> None: _request_context(call_id="call-1", session_id="sess-1", user_id=user_id), pytest.raises(RuntimeError, match="missing user_id"), ): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) def test_hosted_returns_composite_key(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) server.config.is_hosted = True server.config.session_id = "sess-1" with _request_context(call_id="call-1", session_id="sess-1", user_id="user-1"): - assert server._partition_key() == ("sess-1", "user-1") # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == ("sess-1", "user-1") def test_hosted_uses_platform_session_when_context_omits_it(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) server.config.is_hosted = True server.config.session_id = "sess-1" with _request_context(call_id="call-1", user_id="user-1"): - assert server._partition_key() == ("sess-1", "user-1") # pyright: ignore[reportPrivateUsage] + assert invocation_partition_key(server.config) == ("sess-1", "user-1") def test_hosted_rejects_a_different_request_session(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) @@ -1483,7 +1471,7 @@ def test_hosted_rejects_a_different_request_session(self) -> None: _request_context(call_id="call-1", session_id="caller-session", user_id="user-1"), pytest.raises(RuntimeError, match="does not match"), ): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) def test_hosted_requires_platform_call_id(self) -> None: server = InvocationsHostServer(_make_agent(response_text="hi")) @@ -1493,7 +1481,7 @@ def test_hosted_requires_platform_call_id(self) -> None: _request_context(session_id="sess-1", user_id="user-1"), pytest.raises(RuntimeError, match="trusted user ID and call ID"), ): - server._partition_key() # pyright: ignore[reportPrivateUsage] + invocation_partition_key(server.config) async def test_hosted_keys_and_session_ids_preserve_identifier_values(self) -> None: agent = _make_agent(response_text="hi") @@ -1506,7 +1494,7 @@ async def test_hosted_keys_and_session_ids_preserve_identifier_values(self) -> N for session_id, user_id in product(identifiers, repeat=2): server.config.session_id = session_id with _request_context(call_id="call-1", session_id=session_id, user_id=user_id): - key = server._partition_key() # pyright: ignore[reportPrivateUsage] + key = invocation_partition_key(server.config) response = await server._handle_invoke(request) # pyright: ignore[reportPrivateUsage] assert isinstance(key, tuple) @@ -1807,7 +1795,7 @@ async def test_instance_context_lifetime_remains_caller_owned(self) -> None: async def test_factory_agent_context_lifetime_non_streaming(self) -> None: events: list[str] = [] response = AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]) - server = InvocationsHostServer(lambda: _ContextAgent(events, response=response)) + server = InvocationsHostServer(lambda request: _ContextAgent(events, response=response)) with _request_context(session_id="sess-1"): result = await server._handle_invoke(_make_request({"message": "one"})) # pyright: ignore[reportPrivateUsage] @@ -1821,7 +1809,7 @@ async def test_factory_agent_context_lifetime_until_stream_closes(self) -> None: AgentResponseUpdate(contents=[Content.from_text("one")]), AgentResponseUpdate(contents=[Content.from_text("two")]), ] - server = InvocationsHostServer(lambda: _ContextAgent(events, stream_updates=updates)) + server = InvocationsHostServer(lambda request: _ContextAgent(events, stream_updates=updates)) with _request_context(session_id="sess-1"): response = await server._handle_invoke( # pyright: ignore[reportPrivateUsage] @@ -1838,7 +1826,7 @@ async def test_factory_agent_context_lifetime_until_stream_closes(self) -> None: async def test_agent_callable_is_resolved_for_each_request(self) -> None: agents: list[_FakeAgent] = [] - def create_agent() -> _FakeAgent: + def create_agent(*, request: Request) -> _FakeAgent: agent = _make_agent(response_text=f"agent-{len(agents) + 1}") agents.append(agent) return agent @@ -1860,6 +1848,91 @@ def create_agent() -> _FakeAgent: assert agents[0].calls[0]["session"] is not agents[1].calls[0]["session"] assert agents[0].calls[0]["session"].session_id == agents[1].calls[0]["session"].session_id + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("async_factory", [False, True]) + @pytest.mark.parametrize("configured_factory", [False, True]) + async def test_agent_factory_receives_request_after_parsing( + self, stream: bool, async_factory: bool, configured_factory: bool + ) -> None: + requests: list[Request] = [] + agents: list[_FakeAgent] = [] + payloads: list[dict[str, Any]] = [] + + def create_agent(request: Request) -> _FakeAgent: + requests.append(request) + assert request.headers["x-application-context"] == "current-request" + agent = _make_agent(response_text="ok", stream_texts=["ok"]) + agents.append(agent) + return agent + + async def create_agent_async(request: Request) -> _FakeAgent: + payloads.append(await request.json()) + return create_agent(request) + + def configured_agent(model: str = "gpt-4", *, request: Request, temperature: float = 0.7) -> _FakeAgent: + assert model == "gpt-4" + assert temperature == 0.7 + return create_agent(request) + + async def configured_agent_async( + model: str = "gpt-4", *, request: Request, temperature: float = 0.7 + ) -> _FakeAgent: + assert model == "gpt-4" + assert temperature == 0.7 + return await create_agent_async(request) + + factory = ( + (configured_agent_async if async_factory else configured_agent) + if configured_factory + else (create_agent_async if async_factory else create_agent) + ) + server = InvocationsHostServer(factory) + for text in ("first", "second"): + request = Request( + { + "type": "http", + "method": "POST", + "path": "/invocations", + "query_string": b"", + "headers": [(b"x-application-context", b"current-request")], + }, + receive=AsyncMock( + return_value={ + "type": "http.request", + "body": json.dumps({"message": text, "stream": stream}).encode(), + "more_body": False, + } + ), + ) + with _request_context(session_id="sess-1"): + result = await server._handle_invoke(request) + assert await _success_text(result) == "ok" + assert requests[-1] is request + assert len(agents) == 2 + assert agents[0] is not agents[1] + assert payloads == ( + [{"message": text, "stream": stream} for text in ("first", "second")] if async_factory else [] + ) + + @pytest.mark.parametrize("stream", [False, True]) + async def test_factory_type_error_is_not_retried_without_request(self, stream: bool) -> None: + requests: list[Request] = [] + + def create_agent(*, request: Request) -> _FakeAgent: + requests.append(request) + raise TypeError("factory construction failed") + + server = InvocationsHostServer(create_agent) + request = _make_request({"message": "test", "stream": stream}) + with _request_context(session_id="sess-1"): + result = await server._handle_invoke(request) + if stream: + assert isinstance(result, StreamingResponse) + assert [event for event, _ in _parse_sse_events(await _collect_stream(result))] == ["error"] + else: + assert result.status_code == 500 + assert requests == [request] + @pytest.mark.parametrize("stream", [False, True]) @pytest.mark.parametrize("hosted", [False, True]) async def test_reusing_session_restores_identifier(self, hosted: bool, stream: bool) -> None: @@ -1873,7 +1946,9 @@ async def test_reusing_session_restores_identifier(self, hosted: bool, stream: b with ( _request_context(call_id="call-1", session_id="sess-1", user_id="user-1"), - patch("agent_framework_foundry_hosting._invocations.AgentSession", wraps=AgentSession) as session_factory, + patch( + "agent_framework_foundry_hosting._invocations_agent.AgentSession", wraps=AgentSession + ) as session_factory, ): for _ in range(2): response = await server._handle_invoke(request) # pyright: ignore[reportPrivateUsage] diff --git a/python/packages/foundry_hosting/tests/test_request.py b/python/packages/foundry_hosting/tests/test_request.py index 6496e7e8b6f..c7c26826b02 100644 --- a/python/packages/foundry_hosting/tests/test_request.py +++ b/python/packages/foundry_hosting/tests/test_request.py @@ -11,15 +11,15 @@ from azure.ai.agentserver.responses import ResponseContext from azure.ai.agentserver.responses.models import CreateResponse -from agent_framework_foundry_hosting import HostedResponseRequest, InvocationRun, response_input_messages -from agent_framework_foundry_hosting._request import ( +from agent_framework_foundry_hosting import InvocationRun, ResponsesRequest, response_input_messages +from agent_framework_foundry_hosting._scope import FoundryRequestScope +from agent_framework_foundry_hosting._shared import ( prepare_response_options, response_run_options, validate_default_transport_options, validate_request_options, validate_unsupported_options, ) -from agent_framework_foundry_hosting._scope import FoundryRequestScope def test_native_options_are_translated_before_flattened_extra_body() -> None: @@ -82,10 +82,10 @@ async def test_hook_uses_a_request_copy_and_does_not_mutate_defaults() -> None: context.conversation_id = None context.get_input_text = AsyncMock(return_value="hello") scope = FoundryRequestScope(session_id="sandbox", user_id="user", call_id="call", is_hosted=True) - request = HostedResponseRequest(payload, context, scope, {"temperature": 0.8}) + request = ResponsesRequest(payload, context, scope, {"temperature": 0.8}) original = {"temperature": 0.8} - async def remove_temperature(view: HostedResponseRequest, options: dict[str, Any]) -> dict[str, Any]: + async def remove_temperature(view: ResponsesRequest, options: dict[str, Any]) -> dict[str, Any]: assert view.scope is scope assert view.response_id == "outer-id" assert await view.get_input_text() == "hello" @@ -105,7 +105,7 @@ async def test_explicit_message_migration_helper_converts_only_current_turn() -> context.conversation_id = None items = [{"type": "message", "role": "user", "content": [{"type": "input_text", "text": "current"}]}] context.get_input_items = AsyncMock(return_value=items) - request = HostedResponseRequest( + request = ResponsesRequest( cast(CreateResponse, {"input": "current"}), context, FoundryRequestScope("sandbox", "user", "call", True), @@ -113,13 +113,13 @@ async def test_explicit_message_migration_helper_converts_only_current_turn() -> ) converted = [MagicMock()] with patch( - "agent_framework_foundry_hosting._responses._items_to_messages", + "agent_framework_foundry_hosting._responses_shared._items_to_messages", new=AsyncMock(return_value=converted), ) as converter: result = await response_input_messages(request) assert result is converted - converter.assert_awaited_once_with(items, approval_storage=None) + converter.assert_awaited_once_with(items) async def test_hook_rejects_reserved_fields_and_non_mapping_result() -> None: @@ -127,7 +127,7 @@ async def test_hook_rejects_reserved_fields_and_non_mapping_result() -> None: context.response_id = "outer-id" context.conversation_id = None scope = FoundryRequestScope(session_id="sandbox", user_id=None, call_id=None, is_hosted=False) - request = HostedResponseRequest(cast(CreateResponse, {"input": "hello"}), context, scope, {}) + request = ResponsesRequest(cast(CreateResponse, {"input": "hello"}), context, scope, {}) await prepare_response_options(request, lambda _view, _options: {"store": True, "session_id": "forged"}) with pytest.raises(ValueError, match="session_id, store"): diff --git a/python/packages/foundry_hosting/tests/test_responses.py b/python/packages/foundry_hosting/tests/test_responses.py index 1b24defdb31..72cd8b3fd28 100644 --- a/python/packages/foundry_hosting/tests/test_responses.py +++ b/python/packages/foundry_hosting/tests/test_responses.py @@ -16,21 +16,19 @@ import logging import os import uuid -import warnings from collections.abc import AsyncGenerator, AsyncIterator, Awaitable, Callable, Generator, Mapping, Sequence from concurrent.futures import ThreadPoolExecutor -from contextlib import aclosing, asynccontextmanager, suppress +from contextlib import suppress from dataclasses import dataclass from importlib import import_module from pathlib import Path -from typing import Any, Literal, cast, overload +from typing import Any, Literal, cast from unittest.mock import AsyncMock, MagicMock, patch import httpx import pytest from agent_framework import ( Agent, - AgentExecutorRequest, AgentResponse, AgentResponseUpdate, AgentSession, @@ -46,7 +44,6 @@ FinishReasonLiteral, FunctionInvocationLayer, HistoryProvider, - InMemoryCheckpointStorage, InMemoryHistoryProvider, Message, RawAgent, @@ -55,9 +52,6 @@ SessionStore, SupportsAgentRun, WorkflowAgent, - WorkflowBuilder, - WorkflowContext, - executor, tool, ) from agent_framework._mcp import _MCP_TOOL_RESULT_HOST_PAYLOAD_KEY # pyright: ignore[reportPrivateUsage] @@ -90,34 +84,34 @@ from openai.types.responses.response_usage import ResponseUsage as OpenAIResponseUsage from pydantic import TypeAdapter -from agent_framework_foundry_hosting import ResponsesHostServer -from agent_framework_foundry_hosting._responses import ( +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest +from agent_framework_foundry_hosting._responses_shared import ( _INCOMPLETE_REASON_KEY, # pyright: ignore[reportPrivateUsage] _LATEST_CHECKPOINT_ID_KEY, # pyright: ignore[reportPrivateUsage] - CONSENT_ERROR_CODE, - ConsentError, _agent_response_updates, # pyright: ignore[reportPrivateUsage] _annotations_to_output_text, # pyright: ignore[reportPrivateUsage] _await_before_signal, # pyright: ignore[reportPrivateUsage] _container_file_citations_from_function_result, # pyright: ignore[reportPrivateUsage] _create_response_event_stream, # pyright: ignore[reportPrivateUsage] _hosted_internal_metadata, # pyright: ignore[reportPrivateUsage] - _is_allowed_oauth_consent_link, # pyright: ignore[reportPrivateUsage] _item_to_message, # pyright: ignore[reportPrivateUsage] _json_safe_to_str, # pyright: ignore[reportPrivateUsage] - _normalize_allowed_oauth_consent_origins, # pyright: ignore[reportPrivateUsage] _output_item_to_message, # pyright: ignore[reportPrivateUsage] _output_items_to_messages, # pyright: ignore[reportPrivateUsage] _OutputItemTracker, # pyright: ignore[reportPrivateUsage] - _SignalledIterator, # pyright: ignore[reportPrivateUsage] + _PublicResponseFailure, # pyright: ignore[reportPrivateUsage] _stringify_mcp_output, # pyright: ignore[reportPrivateUsage] - consent_url_from_error, ) -from agent_framework_foundry_hosting._state_store import ( +from agent_framework_foundry_hosting._shared import ( AgentSessionStoreProvider, - CheckpointStoreProvider, FoundryAgentSessionStore, - FunctionApprovalStoreProvider, +) +from agent_framework_foundry_hosting._toolbox import ( + CONSENT_ERROR_CODE, + ConsentError, + _is_allowed_oauth_consent_link, + _normalize_allowed_oauth_consent_origins, + consent_url_from_error, ) _OPENAI_HTTPX = cast(Any, import_module(DefaultAsyncHttpxClient.__mro__[1].__module__.partition(".")[0])) @@ -139,12 +133,6 @@ def _isolate_agentserver_state_root(tmp_path_factory: pytest.TempPathFactory) -> os.environ["AGENTSERVER_STATE_ROOT"] = previous_root -def _function_approval_store(request: Content) -> MagicMock: - storage = MagicMock() - storage.load_approval_request = AsyncMock(return_value=request) - return storage - - def _make_function_approval_request_content( *, request_id: str = "apr_test", @@ -335,7 +323,7 @@ def test_container_file_citations_ignore_recursive_json() -> None: }, ) - with patch("agent_framework_foundry_hosting._responses.json.loads", side_effect=RecursionError): + with patch("agent_framework_foundry_hosting._responses_shared.json.loads", side_effect=RecursionError): assert _container_file_citations_from_function_result(function_result) == [] @@ -1588,17 +1576,24 @@ def test_stringify_mcp_output_extracts_only_text_content_mappings(self) -> None: class TestResponsesHostServerInit: + def test_rejects_workflow_agent_before_sdk_construction(self) -> None: + agent = _WorkflowAgentMock() + with ( + patch("azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost.__init__") as initialize, + pytest.raises(TypeError, match="do not support WorkflowAgent"), + ): + ResponsesHostServer(agent) + initialize.assert_not_called() + agent.run.assert_not_called() + @pytest.mark.parametrize("agent", [None, 42]) def test_init_rejects_invalid_agent_source(self, agent: Any) -> None: - with pytest.raises(TypeError, match="agent must be an agent instance or a zero-argument callable"): + with pytest.raises(TypeError, match="agent must be an agent instance or a callable"): ResponsesHostServer(agent) - async def test_zero_argument_agent_class_is_resolved_as_factory(self) -> None: - server = _make_server(cast(Any, _StrictCustomAgent), history_source="agent") - - response = await _post(server) - - assert response.json()["status"] == "completed" + def test_zero_argument_agent_class_is_rejected_as_factory(self) -> None: + with pytest.raises(TypeError, match="required named 'request' parameter"): + _make_server(cast(Any, _StrictCustomAgent), history_source="agent_managed") def test_init_basic(self) -> None: agent = _make_agent( @@ -1623,14 +1618,6 @@ def test_init_uses_default_store_providers(self) -> None: server._session_storage_provider, # pyright: ignore[reportPrivateUsage] AgentSessionStoreProvider, ) - assert isinstance( - server._checkpoint_storage_provider, # pyright: ignore[reportPrivateUsage] - CheckpointStoreProvider, - ) - assert isinstance( - server._function_approval_storage_provider, # pyright: ignore[reportPrivateUsage] - FunctionApprovalStoreProvider, - ) def test_init_rejects_history_provider_with_load_messages(self) -> None: @@ -1666,7 +1653,7 @@ def test_init_allows_history_provider_with_load_messages_for_agent_history(self) ) agent.context_providers = [hp] - ResponsesHostServer(agent, history_source="agent") + ResponsesHostServer(agent, history_source="agent_managed") assert agent.context_providers == [hp] @@ -1694,10 +1681,10 @@ def test_init_allows_default_conversation_id_for_agent_history(self) -> None: default_options={"conversation_id": "service-thread"}, ) - ResponsesHostServer(agent, history_source="agent") + ResponsesHostServer(agent, history_source="agent_managed") def test_init_rejects_custom_agent_for_agent_server_history(self) -> None: - with pytest.raises(RuntimeError, match="history_source='agent'"): + with pytest.raises(RuntimeError, match="history_source='agent_managed'"): ResponsesHostServer(cast(Any, _StrictCustomAgent())) def test_init_requires_storage_capability_for_agent_server_history(self) -> None: @@ -1713,55 +1700,54 @@ def test_failed_init_does_not_mutate_agent(self) -> None: default_options={"store": True}, ) - with pytest.raises(RuntimeError, match="resilient_background"): + with pytest.raises(ValueError, match="resilient_background"): ResponsesHostServer( agent, - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ) assert agent.default_options["store"] is True assert agent.context_providers == [] - def test_init_rejects_resilient_background_for_non_workflow_agent(self, tmp_path: Path) -> None: + def test_init_rejects_background_recovery_without_provider_background(self, tmp_path: Path) -> None: agent = _make_agent( response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("hi")])]) ) - with pytest.raises(RuntimeError, match="resilient_background"): + with pytest.raises(ValueError, match="resilient_background"): ResponsesHostServer( agent, - store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), + response_store=FileResponseStore(storage_dir=tmp_path), + resilient_background=True, ) - def test_init_rejects_steerable_conversations_for_workflow_agent(self) -> None: - workflow_agent = _build_text_workflow_agent("hello from workflow") - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable"): - ResponsesHostServer( - cast(SupportsAgentRun, workflow_agent), - store=InMemoryResponseProvider(), - options=ResponsesServerOptions(steerable_conversations=True), + def test_init_rejects_removed_options(self) -> None: + with pytest.raises(TypeError, match="options"): + cast(Any, ResponsesHostServer)( + _make_agent(), + response_store=InMemoryResponseProvider(), + options=None, ) def test_steering_rejected_before_enabling_task_manager_or_starting_host(self) -> None: agent_factory = MagicMock() def construct(_turn: int) -> str: - with pytest.raises(RuntimeError, match="steerable_conversations=True is temporarily unavailable") as error: - ResponsesHostServer( + with pytest.raises(TypeError, match="steerable_conversations") as error: + cast(Any, ResponsesHostServer)( agent=agent_factory, - options=ResponsesServerOptions(steerable_conversations=True), + steerable_conversations=True, ) return str(error.value) with ( patch("azure.ai.agentserver.core.tasks.set_resilient_tasks_enabled") as enable, - patch("agent_framework_foundry_hosting._responses.ResponsesAgentServerHost.__init__") as base_init, + patch("agent_framework_foundry_hosting._responses_agent.ResponsesAgentServerHost.__init__") as base_init, ThreadPoolExecutor(max_workers=12) as executor, ): failures = list(executor.map(construct, range(24))) assert len(failures) == 24 - assert all("Azure/azure-sdk-for-python#49233" in failure for failure in failures) + assert all("steerable_conversations" in failure for failure in failures) enable.assert_not_called() base_init.assert_not_called() agent_factory.assert_not_called() @@ -1770,7 +1756,7 @@ async def test_non_steerable_background_still_uses_outer_response_id(self) -> No agent = _make_agent( stream_updates=[AgentResponseUpdate(contents=[Content.from_text("finished")], role="assistant")] ) - server = _make_server(agent, options=ResponsesServerOptions(steerable_conversations=False)) + server = _make_server(agent) async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as http: pending = await http.post("/responses", json={"input": "hello", "store": True, "background": True}) @@ -1789,29 +1775,30 @@ async def test_non_steerable_background_still_uses_outer_response_id(self) -> No def test_provider_background_rejects_steering_and_wrong_history_source(self) -> None: agent = Agent(client=_ServiceStorageRecordingClient()) - with pytest.raises(ValueError, match="history_source='service'"): - _make_server(agent, background_source="provider") - with pytest.raises(RuntimeError, match="temporarily unavailable"): + with pytest.raises(ValueError, match="history_source='inference_service'"): + _make_server(agent, background_source="inference_service") + with pytest.raises(TypeError, match="steerable_conversations"): _make_server( agent, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(steerable_conversations=True), + history_source="inference_service", + background_source="inference_service", + steerable_conversations=True, ) - def test_store_alias_warns_once_without_deprecating_history_source(self) -> None: - agent = Agent(client=_ServiceStorageRecordingClient(), default_options=OpenAIChatOptions(store=True)) - with pytest.warns(DeprecationWarning) as recorded: - warnings.warn("unrelated SDK deprecation", DeprecationWarning, stacklevel=2) - server = ResponsesHostServer( - agent, - history_source="agent", - store=InMemoryResponseProvider(), - ) - assert server is not None - messages = [str(warning.message) for warning in recorded] - assert not any(message.startswith("history_source is deprecated;") for message in messages) - assert sum(message.startswith("store= is deprecated;") for message in messages) == 1 + @pytest.mark.parametrize("old_store", [None, InMemoryResponseProvider()]) + @pytest.mark.parametrize("with_response_store", [False, True]) + def test_removed_store_alias_is_rejected_before_sdk_construction( + self, old_store: Any, with_response_store: bool + ) -> None: + agent = _make_agent() + kwargs: dict[str, Any] = {"store": old_store} + if with_response_store: + kwargs["response_store"] = InMemoryResponseProvider() + with patch("azure.ai.agentserver.responses.hosting.ResponsesAgentServerHost.__init__") as initialize: + with pytest.raises(TypeError, match="store"): + ResponsesHostServer(agent, **kwargs) + initialize.assert_not_called() + assert agent.context_providers == [] def test_history_and_background_sources_reject_invalid_values(self) -> None: agent = _make_agent() @@ -1822,10 +1809,24 @@ def test_history_and_background_sources_reject_invalid_values(self) -> None: with pytest.raises(ValueError, match="unsupported_options"): _make_server(agent, unsupported_options="silent") + @pytest.mark.parametrize( + ("setting", "value"), + [ + ("history_source", "agent_server"), + ("history_source", "agent"), + ("history_source", "service"), + ("background_source", "agent_server"), + ("background_source", "provider"), + ], + ) + def test_legacy_source_literals_are_rejected(self, setting: str, value: str) -> None: + with pytest.raises(ValueError, match=setting): + _make_server(_make_agent(), **{setting: value}) + @pytest.mark.parametrize( "identity_field", ["session_id", "agent_session_id", "user_id", "call_id", "service_session_id"] ) - @pytest.mark.parametrize("history_source", ["agent_server", "agent", "service"]) + @pytest.mark.parametrize("history_source", ["host_managed", "agent_managed", "inference_service"]) def test_agent_defaults_cannot_supply_platform_identity(self, identity_field: str, history_source: str) -> None: agent = Agent( client=_ServiceStorageRecordingClient(), @@ -1882,7 +1883,7 @@ async def test_hosted_invalid_identity_fails_without_running_agent( context = ResponseContext(response_id="response-1", mode_flags=MagicMock()) with patch( - "agent_framework_foundry_hosting._responses.get_request_context", + "agent_framework_foundry_hosting._responses_agent.get_request_context", return_value=FoundryAgentRequestContext(session_id=request_session_id, user_id="user-1", call_id=call_id), ): events = [ @@ -1975,6 +1976,19 @@ async def test_responses_history_is_not_duplicated_by_default_local_history(self assert InMemoryHistoryProvider.DEFAULT_SOURCE_ID not in stored.state assert "_foundry_responses_history" not in stored.state + @pytest.mark.parametrize("limit", [1, 3]) + async def test_explicit_history_cap_limits_prior_items_without_dropping_current_input(self, limit: int) -> None: + client = _RecordingHistoryClient() + server = _make_server(Agent(client=client), max_history_items=limit) + + first = await _post(server, input_text="first") + second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) + third = await _post(server, input_text="third", previous_response_id=second.json()["id"]) + + assert third.json()["status"] == "completed", third.json() + prior = ["first", "recorded", "second", "recorded"] + assert [message.text for message in client.calls[-1]] == [*prior[-limit:], "third"] + async def test_agent_server_history_disables_service_storage(self) -> None: client = _ServiceStorageRecordingClient() agent = Agent[ChatOptions]( @@ -2027,7 +2041,7 @@ async def test_host_history_preserves_non_storing_client_default(self) -> None: async def test_agent_history_does_not_forward_runtime_options_to_custom_agent(self) -> None: agent = _StrictCustomAgent() - server = _make_server(agent, session_store=SessionStore(), history_source="agent") + server = _make_server(agent, session_store=SessionStore(), history_source="agent_managed") response = await _post(server, input_text="first", temperature=0.5) @@ -2066,7 +2080,7 @@ async def test_agent_history_preserves_service_storage(self, explicit_store: boo default_options=OpenAIChatOptions(store=True) if explicit_store else None, ) store = SessionStore() - server = _make_server(agent, session_store=store, history_source="agent") + server = _make_server(agent, session_store=store, history_source="agent_managed") first = await _post(server, input_text="first") second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) @@ -2086,7 +2100,7 @@ async def test_service_history_preserves_private_continuation(self) -> None: client = _ServiceStorageRecordingClient() agent = Agent(client=client, default_options=OpenAIChatOptions(store=False)) store = SessionStore() - server = _make_server(agent, session_store=store, history_source="service") + server = _make_server(agent, session_store=store, history_source="inference_service") first = await _post(server, input_text="first") second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) @@ -2103,13 +2117,13 @@ async def test_service_history_preserves_private_continuation(self) -> None: async def test_async_agent_factory_reuses_private_service_session_across_requests(self) -> None: clients: list[_ServiceStorageRecordingClient] = [] - async def create_agent() -> SupportsAgentRun: + async def create_agent(*, request: ResponsesRequest) -> SupportsAgentRun: client = _ServiceStorageRecordingClient() clients.append(client) return Agent(client=client, default_options=OpenAIChatOptions(store=True)) store = SessionStore() - server = _make_server(create_agent, session_store=store, history_source="service") + server = _make_server(create_agent, session_store=store, history_source="inference_service") first = await _post(server, input_text="first") second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) @@ -2122,7 +2136,7 @@ async def create_agent() -> SupportsAgentRun: async def test_service_history_rejects_second_child_of_provider_response(self) -> None: client = _ServiceStorageRecordingClient() - server = _make_server(Agent(client=client), session_store=SessionStore(), history_source="service") + server = _make_server(Agent(client=client), session_store=SessionStore(), history_source="inference_service") first = await _post(server, input_text="first") second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) branch = await _post(server, input_text="fork", previous_response_id=first.json()["id"]) @@ -2135,7 +2149,7 @@ async def test_service_history_rejects_second_child_of_provider_response(self) - async def test_invalid_approval_does_not_consume_service_parent(self) -> None: client = _ServiceStorageRecordingClient() store = SessionStore() - server = _make_server(Agent(client=client), session_store=store, history_source="service") + server = _make_server(Agent(client=client), session_store=store, history_source="inference_service") first = await _post(server, input_text="first") parent_id = first.json()["id"] @@ -2170,7 +2184,7 @@ async def test_agent_history_uses_provider_with_nonstoring_defaults(self) -> Non history = InMemoryHistoryProvider() agent = Agent(client=client, context_providers=[history], default_options=OpenAIChatOptions(store=False)) store = SessionStore() - server = _make_server(agent, session_store=store, history_source="agent") + server = _make_server(agent, session_store=store, history_source="agent_managed") first = await _post(server, input_text="first") second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) @@ -2186,14 +2200,12 @@ async def test_agent_history_uses_provider_with_nonstoring_defaults(self) -> Non assert saved is not None and history.source_id in saved.state assert saved.service_session_id is None - @pytest.mark.parametrize("mode", ["agent_server", "service", "agent"]) + @pytest.mark.parametrize("mode", ["host_managed", "inference_service", "agent_managed"]) async def test_store_false_neither_saves_session_nor_stores_inner_response(self, mode: str) -> None: client = _ServiceStorageRecordingClient() agent = Agent(client=client, default_options=OpenAIChatOptions(store=True)) store = SessionStore() server = _make_server(agent, session_store=store, history_source=mode) - approvals = MagicMock(spec=FunctionApprovalStoreProvider) - server._function_approval_storage_provider = approvals # pyright: ignore[reportPrivateUsage] response = await _post_json(server, {"input": "one shot", "store": False, "model": "test-model"}) @@ -2204,18 +2216,17 @@ async def test_store_false_neither_saves_session_nor_stores_inner_response(self, assert agent.default_options["store"] is True provider = cast(MagicMock, server._session_storage_provider) # pyright: ignore[reportPrivateUsage] provider.get_store.assert_not_called() - approvals.get_store.assert_not_called() async def test_store_false_rejects_external_history_and_custom_agent(self) -> None: history = _PerServiceCallHistoryProvider() agent = Agent(client=_RecordingHistoryClient(), context_providers=[history]) - server = _make_server(agent, history_source="agent") + server = _make_server(agent, history_source="agent_managed") response = await _post_json(server, {"input": "one shot", "store": False}) assert response.json()["status"] == "failed" assert "external HistoryProvider" in response.json()["error"]["message"] custom = _StrictCustomAgent() - server = _make_server(custom, history_source="agent") + server = _make_server(custom, history_source="agent_managed") response = await _post_json(server, {"input": "one shot", "store": False}) assert response.json()["status"] == "failed" assert "custom agent" in response.json()["error"]["message"] @@ -2226,7 +2237,7 @@ async def test_store_false_agent_continuation_fails_instead_of_using_service_his server = _make_server( Agent(client=client, default_options=OpenAIChatOptions(store=True)), session_store=SessionStore(), - history_source="agent", + history_source="agent_managed", ) first = await _post(server) request = CreateResponse(input="cannot resume", store=False, previous_response_id=first.json()["id"]) @@ -2290,7 +2301,7 @@ def prepare_options(_request: Any, _options: dict[str, Any]) -> dict[str, Any]: agent = _make_agent() server = _make_server(agent, prepare_options=prepare_options) - with caplog.at_level(logging.ERROR, logger="agent_framework_foundry_hosting._responses"): + with caplog.at_level(logging.ERROR, logger="agent_framework_foundry_hosting._responses_shared"): response = await _post_json(server, {"input": "one shot", "store": False}) body = response.json() @@ -2343,7 +2354,7 @@ def handle_request(request: Any) -> Any: client=OpenAIChatClient(model="test-model", async_client=openai), default_options=cast(Any, {"extra_body": {"store": True}}), ) - legacy_server = _make_server(legacy_default, history_source="agent") + legacy_server = _make_server(legacy_default, history_source="agent_managed") rejected_default = await _post_json(legacy_server, {"input": "unsafe", "store": False}) assert rejected_default.json()["status"] == "failed" assert "default extra_body" in rejected_default.json()["error"]["message"] @@ -2362,7 +2373,7 @@ def handle_request(request: Any) -> Any: async def test_unsupported_options_policy_rejects_custom_agent(self) -> None: agent = _StrictCustomAgent() - server = _make_server(agent, history_source="agent", unsupported_options="error") + server = _make_server(agent, history_source="agent_managed", unsupported_options="error") response = await _post_json(server, {"input": "hello", "temperature": 0.8}) assert response.json()["status"] == "failed" @@ -2374,7 +2385,7 @@ async def test_custom_agent_unsupported_options_warning_policy( self, policy: str, warn: bool, caplog: pytest.LogCaptureFixture ) -> None: agent = _StrictCustomAgent() - server = _make_server(agent, history_source="agent", unsupported_options=policy) + server = _make_server(agent, history_source="agent_managed", unsupported_options=policy) with caplog.at_level(logging.WARNING): response = await _post(server, temperature=0.5) @@ -2484,9 +2495,9 @@ async def run(messages: Any = None, *, session: AgentSession, **kwargs: Any) -> server = _make_server( agent, session_store=SessionStore(), - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -2544,16 +2555,16 @@ async def run( server = _make_server( agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True, temperature=0.35) context = ResponseContext(response_id="outer-response", mode_flags=MagicMock()) with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): events = [event async for event in server._handle_response(request, context, asyncio.Event())] @@ -2574,7 +2585,7 @@ async def run( stream_updates=[AgentResponseUpdate(contents=[Content.from_text("next")], role="assistant")] ) next_agent.client.STORES_BY_DEFAULT = True - next_server = _make_server(next_agent, session_store=store, history_source="service") + next_server = _make_server(next_agent, session_store=store, history_source="inference_service") next_context = ResponseContext(response_id="outer-next", mode_flags=MagicMock()) with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])): next_events = [ @@ -2618,9 +2629,9 @@ async def run( server = _make_server( agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) context = ResponseContext( @@ -2628,7 +2639,7 @@ async def run( ) with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): events = [ event @@ -2654,9 +2665,9 @@ async def test_provider_recovery_without_saved_token_does_not_restart_job( server = _make_server( agent, session_store=SessionStore(), - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) @@ -2697,9 +2708,9 @@ async def run( first_server = _make_server( first_agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse( @@ -2709,7 +2720,7 @@ async def run( with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), patch( - "agent_framework_foundry_hosting._responses.asyncio.sleep", + "agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock(side_effect=ResponseExitForRecovery()), ), pytest.raises(ResponseExitForRecovery), @@ -2729,16 +2740,16 @@ async def run( resumed_server = _make_server( resumed_agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) recovered_context = ResponseContext(response_id="outer-recover", mode_flags=MagicMock()) recovered_context.is_recovery = True with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): events = [ event async for event in resumed_server._handle_response(request, recovered_context, asyncio.Event()) @@ -2779,9 +2790,9 @@ def make_host() -> ResponsesHostServer: monkeypatch.setattr(agent, "run", MagicMock(side_effect=run)) return _make_server( agent, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -2793,7 +2804,7 @@ def make_host() -> ResponsesHostServer: with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), patch( - "agent_framework_foundry_hosting._responses.asyncio.sleep", + "agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock(side_effect=ResponseExitForRecovery()), ), pytest.raises(ResponseExitForRecovery), @@ -2816,7 +2827,7 @@ def make_host() -> ResponsesHostServer: recovered_host = make_host() with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): events = [ event async for event in recovered_host._handle_response(request, recovered_context, asyncio.Event()) @@ -2865,9 +2876,9 @@ def make_host() -> ResponsesHostServer: monkeypatch.setattr(agent, "run", MagicMock(side_effect=provider_run)) return _make_server( agent, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -2885,7 +2896,7 @@ async def crash_after_head_write(store: FoundryAgentSessionStore, key: str, sess with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), patch.object(FoundryAgentSessionStore, "set", new=crash_after_head_write), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), pytest.raises(ResponseExitForRecovery), ): async for event in server._handle_response(request, context, asyncio.Event()): @@ -2928,7 +2939,7 @@ async def claim_after_recovery_read(reader: FoundryAgentSessionStore, key: str) with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), patch.object(FoundryAgentSessionStore, "get", new=claim_after_recovery_read), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): events = [event async for event in recovered_host._handle_response(request, recovered, asyncio.Event())] @@ -3055,9 +3066,9 @@ async def set(self, session_id: str, session: AgentSession) -> None: server = _make_server( agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) context = ResponseContext(response_id="outer-tool-response", mode_flags=MagicMock()) @@ -3072,7 +3083,7 @@ async def set(self, session_id: str, session: AgentSession) -> None: ), patch.object(client.client.responses.with_raw_response, "create", new=create), patch.object(client.client.responses.with_raw_response, "retrieve", new=retrieve), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): handler = cast( AsyncGenerator[Any, None], @@ -3198,9 +3209,9 @@ async def sleep(_seconds: float) -> None: server = _make_server( agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) response_id = f"outer-{stage}-{interruption}" @@ -3215,7 +3226,7 @@ async def collect_events() -> list[Any]: with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), patch.object(ResponseContext, "exit_for_recovery", new=exit_for_recovery), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=sleep), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=sleep), caplog.at_level(logging.WARNING), ): task = asyncio.create_task(collect_events()) @@ -3314,9 +3325,9 @@ def make_host() -> ResponsesHostServer: monkeypatch.setattr(agent, "run", MagicMock(side_effect=run)) return _make_server( agent, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3326,7 +3337,7 @@ def make_host() -> ResponsesHostServer: response_store = FileResponseStore(storage_dir=tmp_path) with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): handler = cast(AsyncGenerator[Any, None], server._handle_response(request, context, asyncio.Event())) crashed = False @@ -3423,9 +3434,9 @@ def make_host() -> ResponsesHostServer: return _make_server( agent, session_store=store, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) @@ -3440,8 +3451,10 @@ def shutdown_before_output(response: AgentResponse, response_id: str) -> list[Ag with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), patch.object(ResponseContext, "exit_for_recovery", new=AsyncMock(side_effect=ResponseExitForRecovery())), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), - patch("agent_framework_foundry_hosting._responses._agent_response_updates", new=shutdown_before_output), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), + patch( + "agent_framework_foundry_hosting._responses_agent._agent_response_updates", new=shutdown_before_output + ), pytest.raises(ResponseExitForRecovery), ): _ = [event async for event in server._handle_response(request, context, asyncio.Event())] @@ -3458,7 +3471,7 @@ def shutdown_before_output(response: AgentResponse, response_id: str) -> list[Ag recovered.is_recovery = True with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), ): events = [event async for event in make_host()._handle_response(request, recovered, asyncio.Event())] @@ -3500,16 +3513,16 @@ async def run( server = _make_server( agent, session_store=FailingTokenStore() if phase == "save" else SessionStore(), - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, response_store=FileResponseStore(storage_dir=tmp_path), ) request = CreateResponse(input="hello", store=True, background=True) context = ResponseContext(response_id="outer-response", mode_flags=MagicMock()) with ( patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch("agent_framework_foundry_hosting._responses.asyncio.sleep", new=AsyncMock()), + patch("agent_framework_foundry_hosting._responses_agent.asyncio.sleep", new=AsyncMock()), caplog.at_level(logging.WARNING), ): events = [event async for event in server._handle_response(request, context, asyncio.Event())] @@ -3519,7 +3532,7 @@ async def run( assert "private-provider-token" not in str(events) assert "private-provider-token" not in caplog.text for record in caplog.records: - if record.name == "agent_framework_foundry_hosting._responses" and record.exc_info: + if record.name == "agent_framework_foundry_hosting._responses_shared" and record.exc_info: exception = record.exc_info[1] assert exception is not None and exception.__cause__ is None and exception.__context__ is None @@ -3618,7 +3631,7 @@ async def updates() -> AsyncIterator[AgentResponseUpdate]: return ResponseStream(updates(), finalizer=AgentResponse.from_updates) agent.run = MagicMock(side_effect=run) - server = _make_server(agent, history_source="service") + server = _make_server(agent, history_source="inference_service") if existing_head: seed_store = server._session_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] config=server.config, platform_context=get_request_context() @@ -3681,7 +3694,7 @@ async def test_failed_service_conversation_dispatch_keeps_claim(self) -> None: _raising_updates("provider timed out"), finalizer=AgentResponse.from_updates ) ) - server = _make_server(agent, session_store=store, history_source="service") + server = _make_server(agent, session_store=store, history_source="inference_service") conversation_id = "failed-service-conversation" async def collect(response_id: str) -> list[Any]: @@ -3719,7 +3732,7 @@ async def blocked() -> AsyncIterator[AgentResponseUpdate]: yield AgentResponseUpdate(contents=[Content.from_text("too late")], role="assistant") agent.run = MagicMock(side_effect=lambda **kwargs: ResponseStream(blocked())) - server = _make_server(agent, session_store=store, history_source="service") + server = _make_server(agent, session_store=store, history_source="inference_service") conversation_id = "cancelled-service-conversation" context = ResponseContext( response_id="cancelled-dispatch", conversation_id=conversation_id, mode_flags=MagicMock() @@ -3791,7 +3804,7 @@ async def test_conversation_write_conflict_keeps_response_snapshot(self) -> None async def test_http_outer_storage_is_independent_of_inner_service_history(self) -> None: client = _ServiceStorageRecordingClient() store = SessionStore() - server = _make_server(Agent(client=client), history_source="service", session_store=store) + server = _make_server(Agent(client=client), history_source="inference_service", session_store=store) async with httpx.AsyncClient(transport=httpx.ASGITransport(app=server), base_url="http://test") as http: stored = await http.post("/responses", json={"input": "remember me", "store": True}) stored_id = stored.json()["id"] @@ -3808,11 +3821,11 @@ async def test_http_outer_storage_is_independent_of_inner_service_history(self) assert saved is not None and saved.service_session_id == "service-thread-1" assert await store.get(unstored.json()["id"]) is None - @pytest.mark.parametrize("mode", ["service", "agent"]) + @pytest.mark.parametrize("mode", ["inference_service", "agent_managed"]) async def test_http_outer_background_polling_is_independent_of_history_source(self, mode: str) -> None: client = _ServiceStorageRecordingClient() - history = [InMemoryHistoryProvider()] if mode == "agent" else [] - default_options = OpenAIChatOptions(store=False) if mode == "agent" else None + history = [InMemoryHistoryProvider()] if mode == "agent_managed" else [] + default_options = OpenAIChatOptions(store=False) if mode == "agent_managed" else None server = _make_server( Agent(client=client, context_providers=history, default_options=default_options), session_store=SessionStore(), @@ -3833,7 +3846,7 @@ async def test_http_outer_background_polling_is_independent_of_history_source(se assert final.json()["id"] == response_id assert "service-thread-1" not in str(final.json()) - assert client.store_options == ([True] if mode == "service" else [False]) + assert client.store_options == ([True] if mode == "inference_service" else [False]) assert rejected.status_code == 400 async def test_http_background_polling_and_unstored_stream_use_outer_response_id(self) -> None: @@ -3890,7 +3903,7 @@ async def test_agent_history_uses_in_memory_history_from_session_store(self) -> default_options={"store": False}, ) store = SessionStore() - server = _make_server(agent, session_store=store, history_source="agent") + server = _make_server(agent, session_store=store, history_source="agent_managed") first = await _post(server, input_text="first") second = await _post(server, input_text="second", previous_response_id=first.json()["id"]) @@ -5568,8 +5581,7 @@ async def test_function_call_replace_snapshot_extends_fragments(self) -> None: stream_updates=[ AgentResponseUpdate( contents=[ - Content.from_function_call("call_2", "get_weather", - arguments='{"loc', merge_mode="append") + Content.from_function_call("call_2", "get_weather", arguments='{"loc', merge_mode="append") ], role="assistant", ), @@ -6233,11 +6245,11 @@ async def test_mcp_call_with_output_reconstructs_mcp_result_content(self) -> Non assert msg.contents[1].type == "mcp_server_tool_result" assert msg.contents[1].output == "found 10 cats" - async def test_mcp_approval_request(self) -> None: + async def test_mcp_approval_request(self, approval_session: Callable[[Content], AgentSession]) -> None: from azure.ai.agentserver.responses.models import OutputItemMcpApprovalRequest saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = OutputItemMcpApprovalRequest({ "type": "mcp_approval_request", @@ -6246,15 +6258,15 @@ async def test_mcp_approval_request(self) -> None: "name": "dangerous_tool", "arguments": "{}", }) - msg = await _output_item_to_message(item, approval_storage=storage) + msg = await _output_item_to_message(item, session=session) assert msg.role == "assistant" assert msg.contents[0].type == "function_approval_request" - async def test_mcp_approval_response(self) -> None: + async def test_mcp_approval_response(self, approval_session: Callable[[Content], AgentSession]) -> None: from azure.ai.agentserver.responses.models import OutputItemMcpApprovalResponseResource saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = OutputItemMcpApprovalResponseResource({ "type": "mcp_approval_response", @@ -6262,7 +6274,7 @@ async def test_mcp_approval_response(self) -> None: "approval_request_id": "apr-1", "approve": True, }) - msg = await _output_item_to_message(item, approval_storage=storage) + msg = await _output_item_to_message(item, session=session) assert msg.role == "user" assert msg.contents[0].type == "function_approval_response" assert msg.contents[0].approved is True @@ -6865,11 +6877,11 @@ async def test_mcp_call_with_output_reconstructs_mcp_result_content(self) -> Non assert msg.contents[1].type == "mcp_server_tool_result" assert msg.contents[1].output == "found 10 cats" - async def test_mcp_approval_request(self) -> None: + async def test_mcp_approval_request(self, approval_session: Callable[[Content], AgentSession]) -> None: from azure.ai.agentserver.responses.models import ItemMcpApprovalRequest saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = ItemMcpApprovalRequest({ "type": "mcp_approval_request", @@ -6878,23 +6890,23 @@ async def test_mcp_approval_request(self) -> None: "name": "dangerous_tool", "arguments": "{}", }) - msg = await _item_to_message(item, approval_storage=storage) + msg = await _item_to_message(item, session=session) assert msg is not None assert msg.role == "assistant" assert msg.contents[0].type == "function_approval_request" - async def test_mcp_approval_response(self) -> None: + async def test_mcp_approval_response(self, approval_session: Callable[[Content], AgentSession]) -> None: from azure.ai.agentserver.responses.models import MCPApprovalResponse saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = MCPApprovalResponse({ "type": "mcp_approval_response", "approval_request_id": "apr-1", "approve": True, }) - msg = await _item_to_message(item, approval_storage=storage) + msg = await _item_to_message(item, session=session) assert msg is not None assert msg.role == "user" assert msg.contents[0].type == "function_approval_response" @@ -8220,11 +8232,13 @@ async def test_multi_turn_function_call_then_text_and_image(self) -> None: class TestFunctionApprovalConversion: """Tests for the approval-aware paths in `_item_to_message` / `_output_item_to_message`.""" - async def test_output_item_mcp_approval_request_loads_from_storage(self) -> None: + async def test_output_item_mcp_approval_request_uses_native_session( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: from azure.ai.agentserver.responses.models import OutputItemMcpApprovalRequest saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = OutputItemMcpApprovalRequest({ "type": "mcp_approval_request", @@ -8233,17 +8247,17 @@ async def test_output_item_mcp_approval_request_loads_from_storage(self) -> None "name": "dangerous_tool", "arguments": "{}", }) - msg = await _output_item_to_message(item, approval_storage=storage) + msg = await _output_item_to_message(item, session=session) assert msg.role == "assistant" c = msg.contents[0] assert c.type == "function_approval_request" assert c.id == "apr-1" - # The full saved Content (incl. function_call) is restored. + # Wire-supplied names and arguments cannot replace the native pending request. function_call = c.function_call assert function_call is not None assert function_call.name == "delete_file" - async def test_output_item_mcp_approval_request_without_storage_raises(self) -> None: + async def test_output_item_mcp_approval_request_without_session_raises(self) -> None: from azure.ai.agentserver.responses.models import OutputItemMcpApprovalRequest item = OutputItemMcpApprovalRequest({ @@ -8253,14 +8267,16 @@ async def test_output_item_mcp_approval_request_without_storage_raises(self) -> "name": "dangerous_tool", "arguments": "{}", }) - with pytest.raises(ValueError, match="ApprovalStorage is required"): + with pytest.raises(_PublicResponseFailure, match="No native pending approval"): await _output_item_to_message(item) - async def test_output_item_mcp_approval_response_resolves_to_approval_response(self) -> None: + async def test_output_item_mcp_approval_response_resolves_to_approval_response( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: from azure.ai.agentserver.responses.models import OutputItemMcpApprovalResponseResource saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = OutputItemMcpApprovalResponseResource({ "type": "mcp_approval_response", @@ -8268,7 +8284,7 @@ async def test_output_item_mcp_approval_response_resolves_to_approval_response(s "approval_request_id": "apr-1", "approve": True, }) - msg = await _output_item_to_message(item, approval_storage=storage) + msg = await _output_item_to_message(item, session=session) assert msg.role == "user" c = msg.contents[0] assert c.type == "function_approval_response" @@ -8278,7 +8294,7 @@ async def test_output_item_mcp_approval_response_resolves_to_approval_response(s assert function_call is not None assert function_call.name == "delete_file" - async def test_output_item_mcp_approval_response_without_storage_raises(self) -> None: + async def test_output_item_mcp_approval_response_without_session_raises(self) -> None: from azure.ai.agentserver.responses.models import OutputItemMcpApprovalResponseResource item = OutputItemMcpApprovalResponseResource({ @@ -8287,14 +8303,16 @@ async def test_output_item_mcp_approval_response_without_storage_raises(self) -> "approval_request_id": "apr-1", "approve": False, }) - with pytest.raises(ValueError, match="ApprovalStorage is required"): + with pytest.raises(_PublicResponseFailure, match="No native pending approval"): await _output_item_to_message(item) - async def test_input_item_mcp_approval_request_loads_from_storage(self) -> None: + async def test_input_item_mcp_approval_request_uses_native_session( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: from azure.ai.agentserver.responses.models import ItemMcpApprovalRequest saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = ItemMcpApprovalRequest({ "type": "mcp_approval_request", @@ -8303,28 +8321,52 @@ async def test_input_item_mcp_approval_request_loads_from_storage(self) -> None: "name": "dangerous_tool", "arguments": "{}", }) - msg = await _item_to_message(item, approval_storage=storage) + msg = await _item_to_message(item, session=session) assert msg.role == "assistant" assert msg.contents[0].type == "function_approval_request" assert msg.contents[0].id == "apr-1" - async def test_input_item_mcp_approval_response_resolves_to_approval_response(self) -> None: + async def test_input_item_mcp_approval_response_resolves_to_approval_response( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: from azure.ai.agentserver.responses.models import MCPApprovalResponse saved = _make_function_approval_request_content(request_id="apr-1") - storage = _function_approval_store(saved) + session = approval_session(saved) item = MCPApprovalResponse({ "type": "mcp_approval_response", "approval_request_id": "apr-1", "approve": False, }) - msg = await _item_to_message(item, approval_storage=storage) + msg = await _item_to_message(item, session=session) assert msg.role == "user" c = msg.contents[0] assert c.type == "function_approval_response" assert c.approved is False + @pytest.mark.parametrize("request_id, hosted", [("replacement-generation", False), ("provider-approval-id", True)]) + async def test_approval_translation_preserves_native_request_identity( + self, request_id: str, hosted: bool, approval_session: Callable[[Content], AgentSession] + ) -> None: + from agent_framework._tools import _load_pending_approval_requests + from azure.ai.agentserver.responses.models import MCPApprovalResponse + + call = Content.from_function_call( + "provider-call", "delete_file", id="native-call", arguments={"path": "/safe"}, hosted=hosted + ) + pending = Content.from_function_approval_request( + request_id, + call, + additional_properties={"_replacement_approval_request": True} if not hosted else None, + ) + session = approval_session(pending) + item = MCPApprovalResponse(type="mcp_approval_response", approval_request_id=request_id, approve=True) + converted = await _item_to_message(item, session=session) + assert converted.contents[0].id == request_id + assert converted.contents[0].function_call == call + assert set(_load_pending_approval_requests(session)) == {request_id} + @pytest.mark.xdist_group("session_isolation") class TestFunctionApprovalRoundTrip: @@ -8332,16 +8374,26 @@ class TestFunctionApprovalRoundTrip: Turn 1: the agent emits a `function_approval_request` content; the server emits an `mcp_approval_request` output item and persists - the original Content under the emitted id in approval storage. + the native session before publishing it. Turn 2: the caller sends an `mcp_approval_response` input item back; - the server resolves it (via approval storage) into a - `function_approval_response` content delivered to the agent. + the server translates its wire ID and delegates binding to Core, + without consuming the native pending request. """ - async def test_non_streaming_emits_mcp_approval_request_and_persists_to_storage(self) -> None: + @pytest.mark.parametrize("raw_agent", [False, True]) + async def test_non_streaming_emits_mcp_approval_request_and_persists_native_session( + self, raw_agent: bool, approval_session: Callable[[Content], AgentSession] + ) -> None: request_content = _make_function_approval_request_content() - agent = _make_agent(response=AgentResponse(messages=[Message(role="assistant", contents=[request_content])])) - server = _make_server(agent) + agent = _make_agent( + response=AgentResponse(messages=[Message(role="assistant", contents=[request_content])]), + raw_agent=raw_agent, + ) + agent.create_session.side_effect = lambda: approval_session(request_content) + store = SessionStore() + server = _make_server( + agent, session_store=store, history_source="host_managed" if raw_agent else "agent_managed" + ) resp = await _post(server, stream=False) @@ -8354,16 +8406,17 @@ async def test_non_streaming_emits_mcp_approval_request_and_persists_to_storage( assert approval_items[0]["name"] == "delete_file" assert approval_items[0]["server_label"] == "my_server" - # Storage must contain a saved entry under the emitted request id. - loaded = await server._function_approval_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] - config=server.config, platform_context=get_request_context() - ).load_approval_request(approval_request_id) - assert loaded.type == "function_approval_request" - assert loaded.function_call is not None - assert loaded.function_call.name == "delete_file" + assert approval_request_id != request_content.id + assert "internal_metadata" not in approval_items[0] + loaded = await store.get(body["id"]) + assert loaded is not None + assert loaded.to_dict()["state"] == approval_session(request_content).to_dict()["state"] @pytest.mark.parametrize("with_preamble", [False, True]) - async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self, with_preamble: bool) -> None: + @pytest.mark.parametrize("raw_agent", [False, True]) + async def test_streaming_emits_mcp_approval_request_and_persists_native_session( + self, with_preamble: bool, raw_agent: bool, approval_session: Callable[[Content], AgentSession] + ) -> None: request_content = _make_function_approval_request_content(request_id="apr_streaming") updates: list[AgentResponseUpdate] = [] preambles = ("Checking the refund policy.", "The refund requires approval.") @@ -8389,8 +8442,12 @@ async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self ) ) updates.append(AgentResponseUpdate(contents=[request_content], role="assistant")) - agent = _make_agent(stream_updates=updates) - server = _make_server(agent) + agent = _make_agent(stream_updates=updates, raw_agent=raw_agent) + agent.create_session.side_effect = lambda: approval_session(request_content) + store = SessionStore() + server = _make_server( + agent, session_store=store, history_source="host_managed" if raw_agent else "agent_managed" + ) resp = await _post(server, stream=True) @@ -8415,12 +8472,14 @@ async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self break assert approval_request_id is not None - loaded = await server._function_approval_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] - config=server.config, platform_context=get_request_context() - ).load_approval_request(approval_request_id) - assert loaded.type == "function_approval_request" + response_id = events[0]["data"]["response"]["id"] + loaded = await store.get(response_id) + assert loaded is not None + assert loaded.to_dict()["state"] == approval_session(request_content).to_dict()["state"] - async def test_round_trip_approval_response_reaches_agent(self) -> None: + async def test_round_trip_approval_response_reaches_agent( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: """Two-turn: turn 1 emits an approval request; turn 2 sends an approval response and the agent receives a `function_approval_response`.""" request_content = _make_function_approval_request_content() @@ -8431,7 +8490,8 @@ async def test_round_trip_approval_response_reaches_agent(self) -> None: AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("done")])]), ] ) - server = _make_server(agent) + agent.create_session.side_effect = lambda: approval_session(request_content) + server = _make_server(agent, session_store=SessionStore()) first = await _post(server, stream=False) assert first.status_code == 200 @@ -8443,6 +8503,7 @@ async def test_round_trip_approval_response_reaches_agent(self) -> None: # Send back an approval response that references the saved request id. second_payload: dict[str, Any] = { "model": "test-model", + "previous_response_id": first_body["id"], "input": [ { "type": "mcp_approval_response", @@ -8466,7 +8527,9 @@ async def test_round_trip_approval_response_reaches_agent(self) -> None: assert approval_responses[0].approved is True assert approval_responses[0].function_call.name == "delete_file" - async def test_round_trip_approval_response_rejected(self) -> None: + async def test_round_trip_approval_response_rejected( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: """Same as above but the user rejects the approval; the agent must receive `approved=False`.""" request_content = _make_function_approval_request_content() @@ -8477,7 +8540,8 @@ async def test_round_trip_approval_response_rejected(self) -> None: AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("ok")])]), ] ) - server = _make_server(agent) + agent.create_session.side_effect = lambda: approval_session(request_content) + server = _make_server(agent, session_store=SessionStore()) first = await _post(server, stream=False) approval_request_id = next( @@ -8488,6 +8552,7 @@ async def test_round_trip_approval_response_rejected(self) -> None: server, { "model": "test-model", + "previous_response_id": first.json()["id"], "input": [ { "type": "mcp_approval_response", @@ -8530,7 +8595,7 @@ async def test_approval_response_referencing_unknown_id_fails(self) -> None: "stream": False, }, ) - # The handler converts the underlying KeyError into a terminal + # The handler converts the failed native binding into a terminal # ``response.failed`` event, so non-streaming callers see HTTP 200 # with status="failed" and a meaningful error message rather than # a generic 5xx response. @@ -8540,132 +8605,681 @@ async def test_approval_response_referencing_unknown_id_fails(self) -> None: error: dict[str, Any] = body.get("error") or {} assert "apr_unknown" in (error.get("message") or "") + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("approved", [False, True]) + @pytest.mark.parametrize( + "history_source, downstream_store", + [("host_managed", False), ("inference_service", True), ("agent_managed", True), ("agent_managed", False)], + ) + async def test_hosted_approval_replays_only_with_the_owner_of_history( + self, + stream: bool, + approved: bool, + history_source: str, + downstream_store: bool, + hosted_approval_client: tuple[OpenAIChatClient, list[dict[str, Any]]], + ) -> None: + client, calls = hosted_approval_client + agent = Agent[OpenAIChatOptions]( + client=client, + default_options=OpenAIChatOptions(store=downstream_store), + context_providers=[InMemoryHistoryProvider()] if history_source == "agent_managed" else [], + ) + server = _make_server( + agent, + session_store=SessionStore(), + history_source=history_source, + **({"max_history_items": 1} if history_source == "host_managed" else {}), + ) + first = await _post(server, stream=stream) + first_body = _parse_sse_events(first.text)[-1]["data"]["response"] if stream else first.json() + assert first_body["status"] == "completed", first_body + wire_id = next(item["id"] for item in first_body["output"] if item["type"] == "mcp_approval_request") -# endregion + second = await _post_json( + server, + { + "input": [{"type": "mcp_approval_response", "approval_request_id": wire_id, "approve": approved}], + "previous_response_id": first_body["id"], + "stream": stream, + }, + ) + body = _parse_sse_events(second.text)[-1]["data"]["response"] if stream else second.json() + assert body["status"] == "completed", body + assert len(calls) == 2 + controls = [ + item for item in calls[-1]["input"] if item.get("type") in ("mcp_approval_request", "mcp_approval_response") + ] + expected = ( + [] + if downstream_store + else [ + { + "type": "mcp_approval_request", + "id": "provider-approval", + "arguments": '{"path": "/safe"}', + "name": "delete_file", + "server_label": "hosted-tools", + } + ] + ) + assert controls == [ + *expected, + {"type": "mcp_approval_response", "approval_request_id": "provider-approval", "approve": approved}, + ] + assert calls[-1].get("store", True) is downstream_store + assert calls[-1].get("conversation") == ("conv_hosted" if downstream_store else None) + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("history_source", ["host_managed", "agent_managed"]) + @pytest.mark.parametrize( + "arguments, changes", + [ + pytest.param("{}", {"name": "another_tool"}, id="tool"), + pytest.param("{}", {"server_name": "another_server"}, id="server"), + pytest.param('{"path": "/safe"}', {"arguments": '{"path": "/other"}'}, id="arguments"), + pytest.param('{"value": true}', {"arguments": '{"value": 1}'}, id="boolean-to-integer"), + pytest.param('{"value": 1}', {"arguments": '{"value": true}'}, id="integer-to-boolean"), + pytest.param( + '{"value": 1.00000000000000000001}', + {"arguments": '{"value": 1.00000000000000000002}'}, + id="precise-number", + ), + pytest.param("opaque", {"arguments": "different"}, id="opaque-arguments"), + ], + ) + async def test_stale_wire_approval_cannot_bind_to_reused_native_id( + self, + stream: bool, + history_source: str, + arguments: str, + changes: dict[str, str], + approval_session: Callable[[Content], AgentSession], + ) -> None: + from agent_framework._tools import ( + _bind_approval_response_to_pending_request, + _load_pending_approval_requests, + _store_pending_approval_requests, + ) -# region Checkpoint context validation + request = _make_function_approval_request_content(arguments=arguments) + agent = _make_agent(response=AgentResponse(messages=[Message(role="assistant", contents=[request])])) + agent.create_session.side_effect = lambda: approval_session(request) + store = SessionStore() + server = _make_server(agent, session_store=store, history_source=history_source) + first_response = await _post(server, stream=stream) + first = _parse_sse_events(first_response.text)[-1]["data"]["response"] if stream else first_response.json() + assert first["status"] == "completed", first + wire_id = next(item["id"] for item in first["output"] if item["type"] == "mcp_approval_request") + session = await store.get(first["id"]) + assert session is not None + consumed = _bind_approval_response_to_pending_request( + Content("function_approval_response", id=request.id, approved=True), session, consume=True + ) + assert consumed is not None and not _load_pending_approval_requests(session) + replacement = Content.from_dict(request.to_dict()) + assert replacement.function_call is not None + replacement.function_call.id = "new-hosted-occurrence" + replacement.function_call.additional_properties.pop("server_label", None) + for field, value in changes.items(): + setattr(replacement.function_call, field, value) + _store_pending_approval_requests(session, [replacement]) + await store.set(first["id"], session) + response = await _post_json( + server, + { + "input": [{"type": "mcp_approval_response", "approval_request_id": wire_id, "approve": True}], + "previous_response_id": first["id"], + "stream": stream, + }, + ) + body = _parse_sse_events(response.text)[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "failed", body + assert "does not match the current native pending request" in body["error"]["message"] + assert agent.run.call_count == 1 + saved = await store.get(body["id"]) + assert saved is not None and request.id is not None + assert _load_pending_approval_requests(saved)[request.id].to_dict() == replacement.to_dict() -class TestCheckpointContextValidation: + @pytest.mark.parametrize("stream", [False, True]) @pytest.mark.parametrize( - ("context_field", "bad_id"), + "arguments, restored_arguments", [ - ("previous_response_id", "../../escape"), - ("conversation_id", "foo/bar"), - ("response_id", "..\\..\\escape"), + ('{"a": 1, "b": [true, null]}', {"b": [True, None], "a": 1}), + ('{"value": 1.0, "name": "\\u4e2d"}', '{"name": "\u4e2d", "value": 1e0}'), + ("opaque", "opaque"), ], ) - async def test_workflow_rejects_invalid_checkpoint_scope( + async def test_wire_approval_accepts_equivalent_argument_serialization( self, - context_field: str, - bad_id: str, + stream: bool, + arguments: str, + restored_arguments: Any, + approval_session: Callable[[Content], AgentSession], ) -> None: - agent = _WorkflowAgentMock() - agent.run = MagicMock() - agent.context_providers = [] - agent.workflow = MagicMock() - agent.workflow.name = "workflow" - agent.workflow._runner_context.has_checkpointing.return_value = False - server = ResponsesHostServer(agent, store=InMemoryResponseProvider()) - - context_kwargs: dict[str, Any] = {"response_id": "response-current", "mode_flags": MagicMock()} - request = CreateResponse(model="m", input="hi") - if context_field == "previous_response_id": - request = CreateResponse(model="m", input="hi", previous_response_id=bad_id) - context_kwargs["previous_response_id"] = bad_id - else: - context_kwargs[context_field] = bad_id + from agent_framework._tools import _load_pending_approval_requests, _store_pending_approval_requests - with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])): - events = [ - event - async for event in server._handle_response( # pyright: ignore[reportPrivateUsage] - request, - ResponseContext(**context_kwargs), - asyncio.Event(), - ) - ] + request = _make_function_approval_request_content(arguments=arguments) + agent = _make_agent(response=AgentResponse(messages=[Message(role="assistant", contents=[request])])) + agent.create_session.side_effect = lambda: approval_session(request) + store = SessionStore() + server = _make_server(agent, session_store=store) + first = (await _post(server)).json() + wire_id = next(item["id"] for item in first["output"] if item["type"] == "mcp_approval_request") + session = await store.get(first["id"]) + assert session is not None and request.id is not None + restored = _load_pending_approval_requests(session)[request.id] + assert restored.function_call is not None + restored.function_call.arguments = restored_arguments + _store_pending_approval_requests(session, [restored]) + await store.set(first["id"], session) + agent.run.side_effect = _make_agent( + response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("done")])]) + ).run.side_effect - failed_events = [ - event for event in events if isinstance(event, Mapping) and event.get("type") == "response.failed" + response = await _post_json( + server, + { + "input": [{"type": "mcp_approval_response", "approval_request_id": wire_id, "approve": True}], + "previous_response_id": first["id"], + "stream": stream, + }, + ) + body = _parse_sse_events(response.text)[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "completed", body + assert agent.run.call_count == 2 + decisions = [ + content + for message in agent.run.call_args.kwargs["messages"] + for content in message.contents + if content.type == "function_approval_response" ] - assert len(failed_events) == 1 - failed_event = cast(Mapping[str, Any], failed_events[0]) - response = cast(Mapping[str, Any], failed_event["response"]) - error = cast(Mapping[str, Any], response["error"]) - assert "Invalid context id" in error["message"] - agent.run.assert_not_called() + assert len(decisions) == 1 and decisions[0].function_call is not None + assert decisions[0].function_call.arguments == restored_arguments - # endregion + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("limit", ["count", "bytes"]) + @pytest.mark.parametrize("overflow", [False, True], ids=["exact-limit", "over-limit"]) + async def test_deferred_approval_output_has_count_and_byte_limits( + self, + stream: bool, + limit: str, + overflow: bool, + deferred_approval_agent: Callable[ + [Sequence[AgentResponseUpdate], bool], + tuple[MagicMock, list[AgentResponseUpdate], asyncio.Event], + ], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + agent, updates, closed = deferred_approval_agent( + [AgentResponseUpdate(role="assistant", contents=[Content.from_text("\u4e2d" * 32)])], False + ) + total_bytes = sum( + len(update.to_json(ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) + for update in updates + ) + monkeypatch.setattr( + "agent_framework_foundry_hosting._responses_agent._MAX_DEFERRED_APPROVAL_UPDATES", + len(updates) - int(overflow and limit == "count"), + raising=False, + ) + monkeypatch.setattr( + "agent_framework_foundry_hosting._responses_agent._MAX_DEFERRED_APPROVAL_BYTES", + total_bytes - int(overflow and limit == "bytes"), + raising=False, + ) + response = await _post( + _make_server(agent, session_store=SessionStore(), history_source="agent_managed"), stream=stream + ) + events = _parse_sse_events(response.text) if stream else [] + body = events[-1]["data"]["response"] if stream else response.json() + assert body["status"] == ("failed" if overflow else "completed"), body + assert closed.is_set() + approvals = [item for item in body["output"] if item["type"] == "mcp_approval_request"] + if overflow: + assert "Deferred approval output exceeds" in body["error"]["message"] + assert approvals == [] + assert not any(event["data"].get("item", {}).get("type") == "mcp_approval_request" for event in events) + else: + assert len(approvals) == 1 + assert body["output"][-1]["content"][0]["text"] == "\u4e2d" * 32 + @pytest.mark.parametrize("limit", ["count", "bytes"]) + @pytest.mark.parametrize("overflow", [False, True], ids=["exact-limit", "over-limit"]) + async def test_deferred_approval_output_enforces_default_limits( + self, + limit: str, + overflow: bool, + deferred_approval_agent: Callable[ + [Sequence[AgentResponseUpdate], bool], + tuple[MagicMock, list[AgentResponseUpdate], asyncio.Event], + ], + ) -> None: + if limit == "count": + agent, _, closed = deferred_approval_agent( + [AgentResponseUpdate(role="assistant")] * (4095 + int(overflow)), False + ) + else: + text = Content.from_text("") + agent, updates, closed = deferred_approval_agent( + [AgentResponseUpdate(role="assistant", contents=[text])], False + ) + overhead = sum( + len(update.to_json(ensure_ascii=False, allow_nan=False, separators=(",", ":")).encode("utf-8")) + for update in updates + ) + text.text = "x" * (4 * 1024 * 1024 - overhead + int(overflow)) + response = await _post( + _make_server(agent, session_store=SessionStore(), history_source="agent_managed"), stream=False + ) + body = response.json() + assert body["status"] == ("failed" if overflow else "completed"), body["status"] + assert closed.is_set() + if overflow: + assert "Deferred approval output exceeds" in body["error"]["message"] + assert not any(item["type"] == "mcp_approval_request" for item in body["output"]) + else: + assert any(item["type"] == "mcp_approval_request" for item in body["output"]) -# region Agent lifecycle (lazy entry & OAuth consent surfacing) + @pytest.mark.parametrize("stream", [False, True]) + async def test_deferred_approval_limit_closes_an_unending_stream( + self, + stream: bool, + deferred_approval_agent: Callable[ + [Sequence[AgentResponseUpdate], bool], + tuple[MagicMock, list[AgentResponseUpdate], asyncio.Event], + ], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + agent, _, closed = deferred_approval_agent([], True) + monkeypatch.setattr( + "agent_framework_foundry_hosting._responses_agent._MAX_DEFERRED_APPROVAL_UPDATES", 2, raising=False + ) + response = await asyncio.wait_for( + _post(_make_server(agent, session_store=SessionStore(), history_source="agent_managed"), stream=stream), + timeout=5, + ) + events = _parse_sse_events(response.text) if stream else [] + body = events[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "failed", body + assert "update-count limit" in body["error"]["message"] + assert closed.is_set() + assert not any(item["type"] == "mcp_approval_request" for item in body["output"]) + assert not any(event["data"].get("item", {}).get("type") == "mcp_approval_request" for event in events) + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("pending_state", ["missing", "mismatched"]) + async def test_custom_agent_cannot_publish_approval_without_matching_native_state( + self, + stream: bool, + pending_state: str, + approval_session: Callable[[Content], AgentSession], + ) -> None: + from agent_framework._tools import _load_pending_approval_requests -def _make_consent_error( - url: str = "https://consent.example.com/auth", - name: str = "Foundry Toolbox", - source_type: str = "mcp", -) -> Exception: - """Build an exception wrapping a Foundry MCP gateway consent error. + request = _make_function_approval_request_content() + agent = _make_agent( + response=AgentResponse(messages=[Message(role="assistant", contents=[request])]), raw_agent=False + ) + if pending_state == "mismatched": + recorded = Content.from_dict(request.to_dict()) + assert recorded.function_call is not None + recorded.function_call.arguments = '{"path": "/different"}' + agent.create_session.side_effect = lambda: approval_session(recorded) + store = SessionStore() + server = _make_server(agent, session_store=store, history_source="agent_managed") + response = await _post(server, stream=stream) + events = _parse_sse_events(response.text) if stream else [] + body = events[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "failed", body + assert "matching native pending" in body["error"]["message"] + assert not any(item["type"] == "mcp_approval_request" for item in body["output"]) + assert not any(event["data"].get("item", {}).get("type") == "mcp_approval_request" for event in events) + session = await store.get(body["id"]) + assert session is not None + pending = _load_pending_approval_requests(session) + assert len(pending) == (1 if pending_state == "mismatched" else 0) + if pending_state == "mismatched": + assert request.id is not None + pending_call = pending[request.id].function_call + assert pending_call is not None and pending_call.arguments == '{"path": "/different"}' + + @pytest.mark.parametrize("hosted_approval_client", ["failure_after_approval"], indirect=True) + @pytest.mark.parametrize("stream", [False, True]) + async def test_failed_native_finalization_does_not_publish_approval( + self, stream: bool, hosted_approval_client: tuple[OpenAIChatClient, list[dict[str, Any]]] + ) -> None: + from agent_framework._tools import _load_pending_approval_requests - Mirrors the real-world wrapping produced by ``MCPStreamableHTTPTool.__aenter__``, - which catches connection-time ``MCPError``s and re-raises them as a - ``ToolExecutionException`` (an ``AgentFrameworkException`` subclass) with the - original error attached via ``inner_exception``. ``consent_url_from_error`` - then finds the wrapped ``MCPError`` in ``exc.args``. + client, calls = hosted_approval_client + store = SessionStore() + server = _make_server(Agent(client=client), session_store=store) + response = await _post(server, stream=stream) + events = _parse_sse_events(response.text) if stream else [] + body = events[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "failed", body + assert len(calls) == 1 + assert not any(item["type"] == "mcp_approval_request" for item in body["output"]) + assert not any(event["data"].get("item", {}).get("type") == "mcp_approval_request" for event in events) + session = await store.get(body["id"]) + assert session is not None and not _load_pending_approval_requests(session) + + async def test_deferred_approval_preserves_streamed_output_order( + self, approval_session: Callable[[Content], AgentSession] + ) -> None: + request = _make_function_approval_request_content() + agent = _make_agent( + stream_updates=[ + AgentResponseUpdate(role="assistant", contents=[Content.from_text("before")]), + AgentResponseUpdate(role="assistant", contents=[request]), + AgentResponseUpdate(role="assistant", contents=[Content.from_text("after")]), + ] + ) + agent.create_session.side_effect = lambda: approval_session(request) + response = await _post(_make_server(agent, session_store=SessionStore()), stream=True) + events = _parse_sse_events(response.text) + assert events[-1]["data"]["response"]["status"] == "completed" + ordered_output: list[str] = [] + for event in events: + if event["event"] == "response.output_text.done": + ordered_output.append(event["data"]["text"]) + elif ( + event["event"] == "response.output_item.added" + and event["data"]["item"]["type"] == "mcp_approval_request" + ): + ordered_output.append("approval") + assert ordered_output == ["before", "approval", "after"] - The MCPError message uses the structured Foundry MCP gateway format: - a human-readable prefix followed by a JSON document describing each - failed tool source and its consent URL. - """ - from agent_framework.exceptions import ToolExecutionException + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("approved", [False, True]) + @pytest.mark.parametrize( + "history_source, history_limit", + [("host_managed", None), ("host_managed", 1), ("agent_managed", None)], + ) + async def test_native_agent_consumes_serialized_session_approval( + self, + stream: bool, + approved: bool, + history_source: str, + history_limit: int | None, + native_approval_agent: tuple[Agent, list[str], list[list[Message]]], + ) -> None: + from agent_framework._tools import _load_pending_approval_requests - payload = json.dumps({ - "errors": [ - { - "name": name, - "type": source_type, - "error": { - "code": "CONSENT_REQUIRED", - "message": url, - }, - } - ] - }) - message = f"tools/list failed for 1 tool source(s), succeeded for 0 tool source(s) {payload}" - inner = MCPError(CONSENT_ERROR_CODE, message) - return ToolExecutionException("MCP consent required", inner_exception=inner) + agent, executions, calls = native_approval_agent + store = SessionStore() + settings = {"max_history_items": history_limit} if history_source == "host_managed" else {} + server = _make_server(agent, session_store=store, history_source=history_source, **settings) + first = await _post(server, stream=stream) + first_body = _parse_sse_events(first.text)[-1]["data"]["response"] if stream else first.json() + assert first_body["status"] == "completed", first_body + wire_id = next(item["id"] for item in first_body["output"] if item["type"] == "mcp_approval_request") + session = await store.get(first_body["id"]) + assert session is not None + assert set(_load_pending_approval_requests(session)) == {"native-call"} + assert executions == [] + await store.set(first_body["id"], AgentSession.from_dict(json.loads(json.dumps(session.to_dict())))) + decision = {"type": "mcp_approval_response", "approval_request_id": wire_id, "approve": approved} + second = await _post_json( + server, + {"input": [decision], "previous_response_id": first_body["id"], "stream": stream}, + ) + second_body = _parse_sse_events(second.text)[-1]["data"]["response"] if stream else second.json() + assert second_body["status"] == "completed", second_body + assert executions == (["/safe"] if approved else []) + assert len(calls) == 2 + assert any( + content.type == "function_result" and content.call_id == "provider-call" + for message in calls[-1] + for content in message.contents + ) + resumed_calls = [ + content for message in calls[-1] for content in message.contents if content.type == "function_call" + ] + resumed_results = [ + content for message in calls[-1] for content in message.contents if content.type == "function_result" + ] + assert len(resumed_calls) == len(resumed_results) == 1 + assert resumed_calls[0].call_id == resumed_results[0].call_id == "provider-call" + assert not any( + content.type in ("function_approval_request", "function_approval_response") + for message in calls[-1] + for content in message.contents + ) + saved = await store.get(second_body["id"]) + assert saved is not None and not _load_pending_approval_requests(saved) -class TestConsentUrlFromError: - def test_returns_consent_url_when_inner_arg_is_consent_mcp_error(self) -> None: - exc = _make_consent_error("https://example.com/consent", name="my-tool") - assert consent_url_from_error(exc) == [ConsentError(name="my-tool", consent_url="https://example.com/consent")] + third = await _post_json(server, {"input": "continue", "previous_response_id": second_body["id"]}) + assert third.json()["status"] == "completed", third.json() + assert executions == (["/safe"] if approved else []) + assert len(calls) == 3 + assert not any( + content.type in ("function_approval_request", "function_approval_response") + for message in calls[-1] + for content in message.contents + ) - def test_returns_consent_url_for_a2a_preview_source(self) -> None: - exc = _make_consent_error( - "https://example.com/a2a-consent", - name="work-iq", - source_type="a2a_preview", + replay = await _post_json( + server, {"input": [decision], "previous_response_id": second_body["id"], "stream": False} ) - assert consent_url_from_error(exc) == [ - ConsentError(name="work-iq", consent_url="https://example.com/a2a-consent") - ] + assert replay.json()["status"] == "failed" + assert executions == (["/safe"] if approved else []) + assert len(calls) == 3 - def test_returns_none_when_no_mcp_error_in_args(self) -> None: - assert consent_url_from_error(Exception("boom")) is None + @pytest.mark.parametrize("stream", [False, True]) + async def test_wire_correlation_cannot_cross_native_sessions( + self, stream: bool, approval_session: Callable[[Content], AgentSession] + ) -> None: + from agent_framework._tools import _load_pending_approval_requests + + request = _make_function_approval_request_content() + agent = _make_agent(response=AgentResponse(messages=[Message(role="assistant", contents=[request])])) + agent.create_session.side_effect = lambda: approval_session(request) + store = SessionStore() + server = _make_server(agent, session_store=store) + first = (await _post(server)).json() + other = (await _post(server)).json() + wire_id = next(item["id"] for item in first["output"] if item["type"] == "mcp_approval_request") + response = await _post_json( + server, + { + "input": [{"type": "mcp_approval_response", "approval_request_id": wire_id, "approve": True}], + "previous_response_id": other["id"], + "stream": stream, + }, + ) + body = _parse_sse_events(response.text)[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "failed" + assert "does not belong" in body["error"]["message"] + assert agent.run.call_count == 2 + session = await store.get(other["id"]) + assert session is not None + assert set(_load_pending_approval_requests(session)) == {request.id} + + @pytest.mark.parametrize("native_approval_agent", [("/safe", "/other")], indirect=True) + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("first_approved", [False, True]) + async def test_native_approval_batch_waits_for_all_decisions( + self, + stream: bool, + first_approved: bool, + native_approval_agent: tuple[Agent, list[str], list[list[Message]]], + ) -> None: + agent, executions, calls = native_approval_agent + server = _make_server(agent, session_store=SessionStore(), max_history_items=1) + first = await _post(server, stream=stream) + first_body = _parse_sse_events(first.text)[-1]["data"]["response"] if stream else first.json() + wire_ids = { + json.loads(item["arguments"])["path"]: item["id"] + for item in first_body["output"] + if item["type"] == "mcp_approval_request" + } + partial = await _post_json( + server, + { + "input": [ + { + "type": "mcp_approval_response", + "approval_request_id": wire_ids["/safe"], + "approve": first_approved, + } + ], + "previous_response_id": first_body["id"], + "stream": stream, + }, + ) + partial_body = _parse_sse_events(partial.text)[-1]["data"]["response"] if stream else partial.json() + assert partial_body["status"] == "completed", partial_body + assert executions == [] + assert len(calls) == 1 + + final = await _post_json( + server, + { + "input": [ + { + "type": "mcp_approval_response", + "approval_request_id": wire_ids["/other"], + "approve": not first_approved, + } + ], + "previous_response_id": partial_body["id"], + "stream": stream, + }, + ) + final_body = _parse_sse_events(final.text)[-1]["data"]["response"] if stream else final.json() + assert final_body["status"] == "completed", final_body + assert executions == (["/safe"] if first_approved else ["/other"]) + assert len(calls) == 2 + model_calls = [ + content.call_id for message in calls[-1] for content in message.contents if content.type == "function_call" + ] + model_results = [ + content.call_id + for message in calls[-1] + for content in message.contents + if content.type == "function_result" + ] + assert model_calls == model_results == ["provider-call", "provider-call-1"] + + @pytest.mark.parametrize("stream", [False, True]) + async def test_session_save_failure_does_not_publish_resumable_approval( + self, stream: bool, native_approval_agent: tuple[Agent, list[str], list[list[Message]]] + ) -> None: + agent, executions, _ = native_approval_agent + store = SessionStore() + server = _make_server(agent, session_store=store) + with patch.object(store, "set", side_effect=RuntimeError("storage unavailable")): + response = await _post(server, stream=stream) + body = _parse_sse_events(response.text)[-1]["data"]["response"] if stream else response.json() + assert body["status"] == "failed", body + assert not any(item["type"] == "mcp_approval_request" for item in body["output"]) + assert executions == [] + + +# endregion + +# endregion + + +# region Agent lifecycle (lazy entry & OAuth consent surfacing) + + +def _make_consent_error( + url: str = "https://consent.example.com/auth", + name: str = "Foundry Toolbox", + source_type: str = "mcp", +) -> Exception: + """Build an exception wrapping a Foundry MCP gateway consent error. + + Mirrors the real-world wrapping produced by ``MCPStreamableHTTPTool.__aenter__``, + which catches connection-time ``MCPError``s and re-raises them as a + ``ToolExecutionException`` (an ``AgentFrameworkException`` subclass) with the + original error attached via ``inner_exception``. ``consent_url_from_error`` + then finds the wrapped ``MCPError`` in ``exc.args``. + + The MCPError message uses the structured Foundry MCP gateway format: + a human-readable prefix followed by a JSON document describing each + failed tool source and its consent URL. + """ + from agent_framework.exceptions import ToolExecutionException + + payload = json.dumps({ + "errors": [ + { + "name": name, + "type": source_type, + "error": { + "code": "CONSENT_REQUIRED", + "message": url, + }, + } + ] + }) + message = f"tools/list failed for 1 tool source(s), succeeded for 0 tool source(s) {payload}" + inner = MCPError(CONSENT_ERROR_CODE, message) + return ToolExecutionException("MCP consent required", inner_exception=inner) + + +class TestConsentUrlFromError: + def test_returns_consent_url_when_inner_arg_is_consent_mcp_error(self) -> None: + exc = _make_consent_error("https://example.com/consent", name="my-tool") + assert consent_url_from_error(exc) == [ConsentError(name="my-tool", consent_url="https://example.com/consent")] + + def test_returns_consent_url_for_a2a_preview_source(self) -> None: + exc = _make_consent_error( + "https://example.com/a2a-consent", + name="work-iq", + source_type="a2a_preview", + ) + assert consent_url_from_error(exc) == [ + ConsentError(name="work-iq", consent_url="https://example.com/a2a-consent") + ] + + @pytest.mark.parametrize( + "other_error", + [ + {"type": "mcp", "error": {"code": "INTERNAL_ERROR", "message": "Upstream failure"}}, + {"type": "other", "error": {"code": "CONSENT_REQUIRED", "message": "https://example.com/consent"}}, + None, + {"type": "mcp", "error": None}, + {"type": "mcp", "error": {"code": "CONSENT_REQUIRED", "message": 42}}, + { + "name": "", + "type": "mcp", + "error": {"code": "CONSENT_REQUIRED", "message": "https://example.com/consent"}, + }, + ], + ) + @pytest.mark.parametrize("consent_first", [False, True]) + @pytest.mark.parametrize("wrapped", [False, True]) + def test_returns_none_for_mixed_consent_errors( + self, other_error: object, consent_first: bool, wrapped: bool + ) -> None: + from agent_framework.exceptions import ToolExecutionException + + consent = { + "name": "learn", + "type": "mcp", + "error": {"code": "CONSENT_REQUIRED", "message": "https://example.com/consent"}, + } + errors = [consent, other_error] if consent_first else [other_error, consent] + inner = MCPError(CONSENT_ERROR_CODE, "tools/call failed " + json.dumps({"errors": errors})) + exc = ToolExecutionException("MCP failure", inner_exception=inner) if wrapped else inner + + assert consent_url_from_error(exc) is None + + def test_returns_none_when_no_mcp_error_in_args(self) -> None: + assert consent_url_from_error(Exception("boom")) is None def test_returns_none_when_mcp_error_has_different_code(self) -> None: inner = MCPError(-32000, "some other error") exc = Exception("wrapped", inner) assert consent_url_from_error(exc) is None - def test_returns_none_for_bare_mcp_error_without_wrapping(self) -> None: - # `args` of a bare MCPError holds the message string, not an MCPError - # instance, so it does not match the wrapping pattern produced by the - # MCP client when it bubbles consent errors up. + def test_returns_none_for_unstructured_bare_mcp_error(self) -> None: bare = MCPError(CONSENT_ERROR_CODE, "https://x") assert consent_url_from_error(bare) is None @@ -8732,10 +9346,112 @@ def test_invalid_allowlist_origin_raises(self, origin: str) -> None: class TestAgentLifecycle: + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("async_factory", [False, True]) + async def test_rejects_factory_workflow_agent_before_dispatch(self, stream: bool, async_factory: bool) -> None: + agent = _WorkflowAgentMock() + + def create(*, request: ResponsesRequest) -> WorkflowAgent: + return agent + + async def create_async(*, request: ResponsesRequest) -> WorkflowAgent: + return agent + + server = _make_server(create_async if async_factory else create) + result = await _post(server, input_text="hello", stream=stream) + body = _parse_sse_events(result.text)[-1]["data"]["response"] if stream else result.json() + assert body["status"] == "failed" + assert body["output"] == [] + agent.run.assert_not_called() + agent.create_session.assert_not_called() + + @pytest.mark.parametrize("stream", [False, True]) + @pytest.mark.parametrize("async_factory", [False, True]) + @pytest.mark.parametrize("configured_factory", [False, True]) + async def test_factory_receives_current_request_input_scope_and_options( + self, stream: bool, async_factory: bool, configured_factory: bool + ) -> None: + requests: list[ResponsesRequest] = [] + input_texts: list[str | None] = [] + agents: list[MagicMock] = [] + + def create_agent(request: ResponsesRequest) -> MagicMock: + requests.append(request) + assert request.options["temperature"] == 0.7 + assert request.options["max_tokens"] == 17 + assert request.response_id + assert request.scope.session_id + agent = _make_agent(response=AgentResponse(messages=[Message("assistant", request.request["input"])])) + agents.append(agent) + return agent + + async def create_agent_async(request: ResponsesRequest) -> MagicMock: + input_texts.append(await request.get_input_text()) + return create_agent(request) + + def configured_agent(model: str = "gpt-4", *, request: ResponsesRequest, temperature: float = 0.7) -> MagicMock: + assert model == "gpt-4" + assert temperature == 0.7 + return create_agent(request) + + async def configured_agent_async( + model: str = "gpt-4", *, request: ResponsesRequest, temperature: float = 0.7 + ) -> MagicMock: + assert model == "gpt-4" + assert temperature == 0.7 + return await create_agent_async(request) + + factory = ( + (configured_agent_async if async_factory else configured_agent) + if configured_factory + else (create_agent_async if async_factory else create_agent) + ) + server = _make_server( + factory, + prepare_options=lambda request, options: {**options, "temperature": 0.2}, + ) + for text in ("first", "second"): + result = await _post(server, input_text=text, temperature=0.7, max_output_tokens=17, stream=stream) + assert result.status_code == 200 + body = ( + next( + event["data"]["response"] + for event in _parse_sse_events(result.text) + if event["event"] == "response.completed" + ) + if stream + else result.json() + ) + assert body["status"] == "completed" + assert body["output"][0]["content"][0]["text"] == text + + assert [request.request["input"] for request in requests] == ["first", "second"] + assert requests[0] is not requests[1] + assert [request.options["temperature"] for request in requests] == [0.2, 0.2] + assert input_texts == (["first", "second"] if async_factory else []) + assert [agent.__aenter__.await_count for agent in agents] == [1, 1] + assert [agent.__aexit__.await_count for agent in agents] == [1, 1] + + @pytest.mark.parametrize("stream", [False, True]) + async def test_request_factory_type_error_is_not_retried_without_request(self, stream: bool) -> None: + requests: list[ResponsesRequest] = [] + + def create_agent(*, request: ResponsesRequest) -> MagicMock: + requests.append(request) + raise TypeError("factory construction failed") + + result = await _post(_make_server(create_agent), stream=stream) + if stream: + assert any(event["event"] == "response.failed" for event in _parse_sse_events(result.text)) + else: + assert result.json()["status"] == "failed" + assert len(requests) == 1 + assert isinstance(requests[0], ResponsesRequest) + async def test_factory_agent_is_entered_and_exited_for_each_request(self) -> None: agents: list[MagicMock] = [] - def create_agent() -> MagicMock: + def create_agent(*, request: ResponsesRequest) -> MagicMock: agent = _make_agent( response=AgentResponse(messages=[Message(role="assistant", contents=[Content.from_text("hi")])]) ) @@ -9357,16 +10073,6 @@ async def test_forged_internal_metadata_does_not_mark_a_completed_turn_incomplet assert body.get("incomplete_details") is None assert body.get("metadata") == {"source": "client"} - async def test_workflow_agent_content_filter_marks_response_incomplete(self) -> None: - workflow_agent = _build_text_workflow_agent("filtered by workflow", finish_reason="content_filter") - server = _make_server(workflow_agent) - - resp = await _post(server, input_text="hi", stream=False) - assert resp.status_code == 200 - body = resp.json() - assert body["status"] == "incomplete" - assert body["incomplete_details"] == {"reason": "content_filter"} - class TestResponseFailedSurfacing: """Tests that exceptions raised by the hosted agent are converted into @@ -9532,35 +10238,6 @@ def run_failure(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate "total_tokens": 11, } - async def test_workflow_agent_run_failure_emits_response_failed(self) -> None: - """Exceptions raised by a hosted ``WorkflowAgent`` are converted into a - terminal ``response.failed`` event in the same way as the regular - agent path. - """ - workflow_agent = _build_text_workflow_agent("ignored") - private_error_detail = "workflow-secret at C:\\sensitive\\workflow.py" - - def run_failure(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate, AgentResponse]: - del args, kwargs - return ResponseStream( - _raising_updates(private_error_detail), - finalizer=AgentResponse.from_updates, - ) - - # Patch the public ``run`` to fail. ``_handle_inner_workflow`` only - # invokes the agent once (no checkpoint to restore on a fresh - # request), so this is the call that will raise. - with patch.object(workflow_agent, "run", side_effect=run_failure): - server = _make_server(workflow_agent) - resp = await _post(server, input_text="hello", stream=False) - - assert resp.status_code == 200 - body = resp.json() - assert body["status"] == "failed" - error: dict[str, Any] = body.get("error") or {} - assert error.get("message") == "An internal server error occurred." - assert private_error_detail not in resp.text - @pytest.mark.parametrize("sensitive_data", [False, True]) async def test_failure_diagnostics_respect_sensitive_data( self, @@ -9585,7 +10262,7 @@ def run_failure(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate agent.run = MagicMock(side_effect=run_failure) server = _make_server(agent) - with caplog.at_level(logging.ERROR, logger="agent_framework_foundry_hosting._responses"): + with caplog.at_level(logging.ERROR, logger="agent_framework_foundry_hosting._responses_shared"): resp = await _post(server, input_text="hello", stream=True) body = _parse_sse_events(resp.text)[-1]["data"]["response"] @@ -9599,7 +10276,7 @@ def run_failure(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate records = [ record for record in caplog.records - if record.name == "agent_framework_foundry_hosting._responses" + if record.name == "agent_framework_foundry_hosting._responses_shared" and "phase=response production" in record.getMessage() ] assert len(records) == 1 @@ -9608,1049 +10285,83 @@ def run_failure(*args: Any, **kwargs: Any) -> ResponseStream[AgentResponseUpdate # endregion -# region Workflow agent hosting (end-to-end) - - -class _ToolApprovalWorkflowAgentMock(SupportsAgentRun): - """Inner agent for a hosted ``WorkflowAgent`` whose first run emits a - ``FunctionApprovalRequestContent`` and whose follow-up run (after - receiving a ``FunctionApprovalResponseContent`` in its inputs) returns a - final assistant text response. +# region Parallel pre-model reads (_load_request_messages) - Mirrors a real agent whose tool invocation requires user approval. Used - here to exercise the full HTTP pipeline through ``ResponsesHostServer`` - when the hosted agent is a ``WorkflowAgent`` containing a tool-approval - flow. - """ - def __init__( - self, - name: str, - *, - tool_name: str = "delete_file", - tool_arguments: dict[str, Any] | None = None, - approval_request_ids: Sequence[str] | None = None, - final_text: str = "done", - ) -> None: - self.id = str(uuid.uuid4()) - self.name = name - self.description: str | None = None - self._tool_name = tool_name - self._tool_arguments = tool_arguments or {"path": "/tmp/example"} - self._approval_request_ids: list[str] = list(approval_request_ids) if approval_request_ids else [] - self._final_text = final_text - self.run_count = 0 - self.last_run_messages: list[Message] = [] - - def create_session(self, **kwargs: Any) -> AgentSession: - del kwargs - return AgentSession() - - def get_session(self, service_session_id: str | ServiceSessionId, *, session_id: str | None = None) -> AgentSession: - del service_session_id, session_id - return AgentSession() - - def _next_request_id(self) -> str: - # Stable across calls: when the workflow checkpoint round-trips through - # restore, ``AgentExecutor`` re-invokes the inner agent during replay. - # We must surface the *same* approval request id on each invocation so - # the workflow's pending-request id matches the id the test echoes - # back as ``mcp_approval_response``. - if self._approval_request_ids: - return self._approval_request_ids[0] - return str(uuid.uuid4()) - - def _build_approval_request(self) -> Content: - request_id = self._next_request_id() - function_call = Content.from_function_call( - call_id=request_id, - name=self._tool_name, - arguments=self._tool_arguments, - additional_properties={"server_label": "test_server"}, - ) - return Content.from_function_approval_request(id=request_id, function_call=function_call) - - @overload - def run( - self, - messages: str | Content | Message | Sequence[str | Content | Message] | None = ..., - *, - stream: Literal[False] = ..., - session: AgentSession | None = ..., - **kwargs: Any, - ) -> Awaitable[AgentResponse[Any]]: ... +class TestParallelRequestReads: + """Covers the concurrent input/history read helper introduced to remove serial + latency from the request critical path (overlap, ordering, and no orphaned + storage reads when one side fails).""" - @overload - def run( - self, - messages: str | Content | Message | Sequence[str | Content | Message] | None = ..., - *, - stream: Literal[True], - session: AgentSession | None = ..., - **kwargs: Any, - ) -> ResponseStream[AgentResponseUpdate, AgentResponse[Any]]: ... + @staticmethod + def _identity_converters(monkeypatch: pytest.MonkeyPatch) -> None: + async def _passthrough( + items: Any, + *, + session: Any = None, + approval_ids: Any = None, + include_hosted_approval_requests: bool = False, + ) -> list[Any]: + del session, approval_ids, include_hosted_approval_requests + return list(items) - def run( - self, - messages: str | Content | Message | Sequence[str | Content | Message] | None = None, - *, - stream: bool = False, - session: AgentSession | None = None, - **kwargs: Any, - ) -> Awaitable[AgentResponse] | ResponseStream[AgentResponseUpdate, AgentResponse]: - del session - assert stream is True, "The inner agent only runs in stream mode in Foundry Hosted Agents." - return self._run_stream(messages=messages, **kwargs) + monkeypatch.setattr("agent_framework_foundry_hosting._responses_agent._items_to_messages", _passthrough) + monkeypatch.setattr("agent_framework_foundry_hosting._responses_agent._output_items_to_messages", _passthrough) - @staticmethod - def _normalize( - messages: str | Content | Message | Sequence[str | Content | Message] | None, - ) -> list[Message]: - if messages is None: - return [] - if isinstance(messages, str): - return [Message(role="user", contents=[Content.from_text(text=messages)])] - if isinstance(messages, Message): - return [messages] - if isinstance(messages, Content): - return [Message(role="user", contents=[messages])] - result: list[Message] = [] - for item in messages: - if isinstance(item, Message): - result.append(item) - elif isinstance(item, Content): - result.append(Message(role="user", contents=[item])) - else: - result.append(Message(role="user", contents=[Content.from_text(text=item)])) - return result + async def test_reads_overlap_and_preserve_order(self, monkeypatch: pytest.MonkeyPatch) -> None: + self._identity_converters(monkeypatch) + server = _make_server(_make_agent()) + server._uses_agent_server_history = True # pyright: ignore[reportPrivateUsage] - @staticmethod - def _approval_responses_in(messages: list[Message]) -> list[Content]: - return [c for m in messages for c in m.contents if c.type == "function_approval_response"] + history_msg = cast(OutputItem, {"type": "output_message", "role": "assistant", "content": []}) + input_msg = cast(Item, {"type": "message", "role": "user", "content": "I"}) - def _run_stream( - self, - messages: str | Content | Message | Sequence[str | Content | Message] | None = None, - **kwargs: Any, - ) -> ResponseStream[AgentResponseUpdate, AgentResponse]: - del kwargs - normalized = self._normalize(messages) - self.last_run_messages = normalized - self.run_count += 1 - approvals = self._approval_responses_in(normalized) + input_started = asyncio.Event() + history_started = asyncio.Event() + release = asyncio.Event() - async def _iter() -> AsyncIterator[AgentResponseUpdate]: - if approvals: - yield AgentResponseUpdate( - contents=[Content.from_text(text=self._final_text)], - role="assistant", - author_name=self.name, - ) - return - yield AgentResponseUpdate( - contents=[self._build_approval_request()], - role="assistant", - author_name=self.name, - ) + async def get_input_items() -> list[Item]: + input_started.set() + await release.wait() + return [input_msg] - return ResponseStream(_iter(), finalizer=AgentResponse.from_updates) + async def get_history() -> list[OutputItem]: + history_started.set() + await release.wait() + return [history_msg] + context = MagicMock(spec=ResponseContext) + context.get_input_items = get_input_items + context.get_history = get_history -def _build_text_workflow_agent(text: str, *, finish_reason: FinishReasonLiteral | None = None) -> WorkflowAgent: - """Build a minimal ``WorkflowAgent`` whose inner agent emits a fixed text.""" + task = asyncio.ensure_future(server._load_request_messages(context)) # pyright: ignore[reportPrivateUsage] + try: + # Both reads must be in-flight before either is allowed to finish — proves they overlap. + await asyncio.wait_for(input_started.wait(), timeout=1) + await asyncio.wait_for(history_started.wait(), timeout=1) + release.set() + messages = await asyncio.wait_for(task, timeout=1) + finally: + release.set() - class _TextAgent(SupportsAgentRun): - def __init__(self, name: str, text: str, finish_reason: FinishReasonLiteral | None) -> None: - self.id = str(uuid.uuid4()) - self.name = name - self.description: str | None = None - self._text = text - self._finish_reason: FinishReasonLiteral | None = finish_reason + # History precedes input in the assembled model input, and the helper owns the ordering. + assert messages == [history_msg, input_msg] - def create_session(self, **kwargs: Any) -> AgentSession: - del kwargs - return AgentSession() + async def test_history_read_skipped_without_agent_server_history(self, monkeypatch: pytest.MonkeyPatch) -> None: + self._identity_converters(monkeypatch) + server = _make_server(_make_agent()) + server._uses_agent_server_history = False # pyright: ignore[reportPrivateUsage] - def get_session( - self, service_session_id: str | ServiceSessionId, *, session_id: str | None = None - ) -> AgentSession: - del service_session_id, session_id - return AgentSession() + input_msg = cast(Item, {"type": "message", "role": "user", "content": "I"}) - @overload - def run( - self, - messages: Any = ..., - *, - stream: Literal[False] = ..., - session: AgentSession | None = ..., - **kwargs: Any, - ) -> Awaitable[AgentResponse[Any]]: ... - - @overload - def run( - self, - messages: Any = ..., - *, - stream: Literal[True], - session: AgentSession | None = ..., - **kwargs: Any, - ) -> ResponseStream[AgentResponseUpdate, AgentResponse[Any]]: ... - - def run( - self, - messages: Any = None, - *, - stream: bool = False, - session: AgentSession | None = None, - **kwargs: Any, - ) -> Awaitable[AgentResponse] | ResponseStream[AgentResponseUpdate, AgentResponse]: - del messages, session, kwargs - assert stream is True, "The inner agent only runs in stream mode in Foundry Hosted Agents." - text = self._text - name = self.name - finish_reason = self._finish_reason - - async def _aiter() -> AsyncIterator[AgentResponseUpdate]: - yield AgentResponseUpdate( - contents=[Content.from_text(text=text)], - role="assistant", - author_name=name, - finish_reason=finish_reason, - ) - - return ResponseStream(_aiter(), finalizer=AgentResponse.from_updates) - - inner = _TextAgent("text-agent", text, finish_reason) - - @executor - async def start(messages: list[Message], ctx: WorkflowContext[AgentExecutorRequest]) -> None: - await ctx.send_message(AgentExecutorRequest(messages=messages, should_respond=True)) - - workflow = WorkflowBuilder(start_executor=start).add_edge(start, inner).build() - return WorkflowAgent(workflow=workflow, name="Text Workflow Agent") - - -class _MultiUpdateWorkflowAgentMock(SupportsAgentRun): - """Inner agent that streams one update per text in a single ``run`` call, and tracks ``run_count``.""" - - def __init__(self, name: str, texts: Sequence[str], *, gate: asyncio.Event | None = None) -> None: - self.id = str(uuid.uuid4()) - self.name = name - self.description: str | None = None - self._texts = list(texts) - self._gate = gate - self.run_count = 0 - self.started = asyncio.Event() - self.cancelled = asyncio.Event() - - def create_session(self, **kwargs: Any) -> AgentSession: - del kwargs - return AgentSession() - - def get_session(self, service_session_id: str | ServiceSessionId, *, session_id: str | None = None) -> AgentSession: - del service_session_id, session_id - return AgentSession() - - @overload - def run( - self, - messages: Any = ..., - *, - stream: Literal[False] = ..., - session: AgentSession | None = ..., - **kwargs: Any, - ) -> Awaitable[AgentResponse[Any]]: ... - - @overload - def run( - self, - messages: Any = ..., - *, - stream: Literal[True], - session: AgentSession | None = ..., - **kwargs: Any, - ) -> ResponseStream[AgentResponseUpdate, AgentResponse[Any]]: ... - - def run( - self, - messages: Any = None, - *, - stream: bool = False, - session: AgentSession | None = None, - **kwargs: Any, - ) -> Awaitable[AgentResponse] | ResponseStream[AgentResponseUpdate, AgentResponse]: - del messages, session, kwargs - assert stream is True, "The inner agent only runs in stream mode in Foundry Hosted Agents." - self.run_count += 1 - texts = self._texts - name = self.name - gate = self._gate - - async def _aiter() -> AsyncIterator[AgentResponseUpdate]: - self.started.set() - if gate is not None: - try: - await gate.wait() # Simulates a stuck model/tool call for preemption tests. - except asyncio.CancelledError: - self.cancelled.set() - raise - for text in texts: - yield AgentResponseUpdate( - contents=[Content.from_text(text=text)], - role="assistant", - author_name=name, - ) - - return ResponseStream(_aiter(), finalizer=AgentResponse.from_updates) - - -def _build_multi_update_workflow_agent( - texts: Sequence[str], *, gate: asyncio.Event | None = None -) -> tuple[WorkflowAgent, _MultiUpdateWorkflowAgentMock]: - """Build a ``WorkflowAgent`` whose inner agent streams one update per text in ``texts``.""" - inner = _MultiUpdateWorkflowAgentMock("multi-update-agent", texts, gate=gate) - - @executor - async def start(messages: list[Message], ctx: WorkflowContext[AgentExecutorRequest]) -> None: - await ctx.send_message(AgentExecutorRequest(messages=messages, should_respond=True)) - - workflow = WorkflowBuilder(name="multi-update-workflow", start_executor=start).add_edge(start, inner).build() - return WorkflowAgent(workflow=workflow, name="Multi Update Workflow Agent"), inner - - -@asynccontextmanager -async def _pending_workflow_event( - handler: AsyncGenerator[Any], started: asyncio.Event -) -> AsyncGenerator[asyncio.Future[Any]]: - pending = asyncio.ensure_future(anext(handler)) - started_wait = asyncio.ensure_future(started.wait()) - try: - # Startup uses pytest's test timeout; only preemption has a short deadline. - await asyncio.wait([pending, started_wait], return_when=asyncio.FIRST_COMPLETED) - if pending.done(): - pytest.fail(f"Workflow returned before reaching the blocked call: {pending.result()!r}") - yield pending - finally: - started_wait.cancel() - pending.cancel() - await asyncio.gather(started_wait, pending, return_exceptions=True) - await handler.aclose() - - -def _build_approval_workflow_agent( - *, - approval_request_id: str, - tool_name: str = "delete_file", - tool_arguments: dict[str, Any] | None = None, - final_text: str = "done", -) -> tuple[WorkflowAgent, _ToolApprovalWorkflowAgentMock]: - """Build a ``WorkflowAgent`` whose inner agent emits a tool approval request.""" - mock_agent = _ToolApprovalWorkflowAgentMock( - name="approval-agent", - tool_name=tool_name, - tool_arguments=tool_arguments or {"path": "/tmp/secret.txt"}, - approval_request_ids=[approval_request_id], - final_text=final_text, - ) - - @executor - async def start(messages: list[Message], ctx: WorkflowContext[AgentExecutorRequest]) -> None: - await ctx.send_message(AgentExecutorRequest(messages=messages, should_respond=True)) - - workflow = WorkflowBuilder(start_executor=start).add_edge(start, mock_agent).build() - workflow_agent = WorkflowAgent(workflow=workflow, name="Approval Workflow Agent") - return workflow_agent, mock_agent - - -class TestWorkflowAgentHosting: - """End-to-end HTTP tests for ``ResponsesHostServer`` hosting a ``WorkflowAgent``. - - These tests drive ``_handle_inner_workflow`` through the ASGI stack: - they exercise checkpoint write/restore (multi-turn) and the - tool-approval round-trip path, which is the primary differentiator - relative to the regular agent path. - """ - - async def test_async_factory_creates_workflow_agent_for_each_request(self) -> None: - created: list[tuple[WorkflowAgent, _MultiUpdateWorkflowAgentMock]] = [] - - async def create_agent() -> WorkflowAgent: - agent, inner = _build_multi_update_workflow_agent(["hello"]) - created.append((agent, inner)) - return agent - - server = _make_server(create_agent) - - first = await _post(server, input_text="one") - second = await _post(server, input_text="two") - - assert first.status_code == 200 - assert second.status_code == 200 - assert len(created) == 2 - assert created[0][0].workflow is not created[1][0].workflow - assert [inner.run_count for _, inner in created] == [1, 1] - - async def test_factory_workflow_restores_checkpoint_for_same_conversation(self) -> None: - runs: list[MagicMock] = [] - - def create_agent() -> WorkflowAgent: - agent, _ = _build_multi_update_workflow_agent(["hello"]) - run = MagicMock(wraps=agent.run) - cast(Any, agent).run = run - runs.append(run) - return agent - - checkpoint_storage = InMemoryCheckpointStorage() - checkpoint_provider = MagicMock(spec=CheckpointStoreProvider) - checkpoint_provider.get_store.return_value = checkpoint_storage - server = _make_server(create_agent, checkpoint_store_provider=checkpoint_provider) - - first = await _post(server, input_text="one", conversation_id="conversation-1") - second = await _post(server, input_text="two", conversation_id="conversation-1") - - assert first.status_code == 200 - assert second.status_code == 200 - assert [run.call_count for run in runs] == [1, 2] - - async def test_basic_text_response(self) -> None: - workflow_agent = _build_text_workflow_agent("hello from workflow") - server = _make_server(workflow_agent) - - resp = await _post(server, input_text="hi", stream=False) - assert resp.status_code == 200 - body = resp.json() - assert body["status"] == "completed" - - text_found = any( - part.get("type") == "output_text" and part.get("text") == "hello from workflow" - for item in body["output"] - if item["type"] == "message" - for part in item.get("content", []) - ) - assert text_found, f"Expected workflow output text in {body['output']}" - - async def test_basic_text_response_streaming(self) -> None: - workflow_agent = _build_text_workflow_agent("hello stream") - server = _make_server(workflow_agent) - - resp = await _post(server, input_text="hi", stream=True) - assert resp.status_code == 200 - events = _parse_sse_events(resp.text) - types = _sse_event_types(events) - assert types[0] == "response.created" - assert types[-1] == "response.completed" - assert "response.output_text.delta" in types - text_done = [e for e in events if e["event"] == "response.output_text.done"] - assert any(e["data"]["text"] == "hello stream" for e in text_done) - - async def test_cancellation_signal_stops_main_loop_without_completing(self) -> None: - """Explicit-cancel: the workflow's main loop must break promptly, and the handler must not - emit a ``response.completed`` terminal for a run it didn't finish (regression for #8564).""" - workflow_agent, inner = _build_multi_update_workflow_agent(["one", "two", "three"]) - server = _make_server(workflow_agent) - request = CreateResponse(model="m", input="hi", stream=True) - context = ResponseContext(response_id="response-1", mode_flags=MagicMock()) - cancellation_signal = asyncio.Event() - - with ( - patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), - ): - handler = cast( - AsyncGenerator[Any, None], - server._handle_response(request, context, cancellation_signal), # pyright: ignore[reportPrivateUsage] - ) - events: list[Any] = [] - async for event in handler: - events.append(event) - if isinstance(event, Mapping) and event.get("type") == "response.output_text.delta": - break - # Cancellation arrives after the first delta, via the explicit /cancel endpoint (both - # the signal and its cause flag fire together); the loop must not process "two"/"three". - context.client_cancelled = True - cancellation_signal.set() - events.extend([event async for event in handler]) - - types = [event.get("type") for event in events if isinstance(event, Mapping)] - assert types.count("response.output_text.delta") == 1 - assert "response.completed" not in types - assert types[-1] == "response.output_text.delta" - assert inner.run_count == 1 - - async def test_cancellation_signal_preempts_stuck_workflow_call(self) -> None: - """Explicit-cancel must interrupt the workflow's inner agent call stuck awaiting a slow - model/tool response, not merely be checked between already-produced updates.""" - gate = asyncio.Event() # Never set: simulates a model/tool call that never returns. - workflow_agent, inner = _build_multi_update_workflow_agent(["too late"], gate=gate) - server = _make_server(workflow_agent) - request = CreateResponse(model="m", input="hi", stream=True) - context = ResponseContext(response_id="response-1", mode_flags=MagicMock()) - cancellation_signal = asyncio.Event() - - with ( - patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), - ): - handler = cast( - AsyncGenerator[Any, None], - server._handle_response(request, context, cancellation_signal), # pyright: ignore[reportPrivateUsage] - ) - await anext(handler) # response.created - await anext(handler) # response.in_progress - - async with _pending_workflow_event(handler, inner.started) as pending: - context.client_cancelled = True - cancellation_signal.set() # Fires while the inner agent is stuck awaiting `gate`. - - async def _drain() -> list[Any]: - events: list[Any] = [] - try: - events.append(await pending) - except StopAsyncIteration: - return events - events.extend([event async for event in handler]) - return events - - # Bounded well below `gate` never being set: proves cancellation preempted the stuck - # call instead of only being observed after it (eventually) produced an update. - events = await asyncio.wait_for(_drain(), timeout=1.0) - assert inner.cancelled.is_set() - - types = [event.get("type") for event in events if isinstance(event, Mapping)] - assert "response.output_text.delta" not in types - assert "response.completed" not in types - assert inner.run_count == 1 - - async def test_shutdown_signal_preempts_stuck_workflow_call(self, tmp_path: Path) -> None: - """Shutdown must interrupt the workflow's inner agent call stuck awaiting a slow model/tool - response, not merely be checked between already-produced updates, and must trigger - ``exit_for_recovery()`` because it actually preempted the loop -- not on natural completion.""" - gate = asyncio.Event() # Never set: simulates a model/tool call that never returns. - workflow_agent, inner = _build_multi_update_workflow_agent(["too late"], gate=gate) - server = _make_server( - workflow_agent, - response_store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), - ) - request = CreateResponse(model="m", input="hi", stream=True) - context = ResponseContext(response_id="response-1", mode_flags=MagicMock()) - cancellation_signal = asyncio.Event() - - with ( - patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "exit_for_recovery", new=AsyncMock(side_effect=ResponseExitForRecovery())), - ): - handler = cast( - AsyncGenerator[Any, None], - server._handle_response(request, context, cancellation_signal), # pyright: ignore[reportPrivateUsage] - ) - await anext(handler) # response.created - await anext(handler) # response.in_progress - - async with _pending_workflow_event(handler, inner.started) as pending: - context.shutdown.set() # Fires while the inner agent is stuck awaiting `gate`. - - # Bounded well below `gate` never being set: proves shutdown preempted the stuck call - # instead of only being observed after it (eventually) produced an update. - with pytest.raises(ResponseExitForRecovery): - await asyncio.wait_for(pending, timeout=1.0) - assert inner.cancelled.is_set() - - assert inner.run_count == 1 - - async def test_cancellation_signal_set_before_turn_skips_new_input(self) -> None: - """Explicit-cancel: cancellation set before a continuation turn starts must skip that turn's new - input entirely, whether caught by the restore-loop's own check or the standalone check - guarding the start of a brand new workflow run.""" - workflow_agent, inner = _build_multi_update_workflow_agent(["hello"]) - server = _make_server(workflow_agent) - - first = await _post(server, conversation_id="conv-1", stream=False) - assert first.status_code == 200 - run_count_after_first_turn = inner.run_count - assert run_count_after_first_turn == 1 - - request = CreateResponse(model="m", input="hi again", stream=True) - context = ResponseContext(response_id="response-2", mode_flags=MagicMock(), conversation_id="conv-1") - cancellation_signal = asyncio.Event() - context.client_cancelled = True # Explicit cancel already present before the turn even starts. - cancellation_signal.set() - - with ( - patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), - ): - handler = cast( - AsyncGenerator[Any, None], - server._handle_response(request, context, cancellation_signal), # pyright: ignore[reportPrivateUsage] - ) - events = [event async for event in handler] - - types = [event.get("type") for event in events if isinstance(event, Mapping)] - assert "response.output_text.delta" not in types - assert "response.completed" not in types - assert types[-1] == "response.in_progress" - # At most the restore-only replay call happened; the new-turn call (which would deliver - # "hi again") must never fire. - assert inner.run_count <= run_count_after_first_turn + 1 - - async def test_shutdown_signal_set_before_restore_only_triggers_recovery(self, tmp_path: Path) -> None: - """Shutdown observed while resuming a checkpoint (whether during the restore-only replay or - the standalone check guarding the start of a brand new workflow run) must trigger - ``exit_for_recovery()`` -- proving the post-loop ``signalled`` check (not a blind re-check of - the flag) correctly gates this action so it doesn't also fire on a replay that merely - finished naturally.""" - workflow_agent, inner = _build_multi_update_workflow_agent(["hello"]) - server = _make_server( - workflow_agent, - response_store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), - ) - - first = await _post(server, conversation_id="conv-1", stream=False) - assert first.status_code == 200 - run_count_after_first_turn = inner.run_count - assert run_count_after_first_turn == 1 - - request = CreateResponse(model="m", input="hi again", stream=True) - context = ResponseContext(response_id="response-2", mode_flags=MagicMock(), conversation_id="conv-1") - cancellation_signal = asyncio.Event() - context.shutdown.set() # Fires before the continuation turn even starts. - - with ( - patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "get_history", new=AsyncMock(return_value=[])), - patch.object(ResponseContext, "exit_for_recovery", new=AsyncMock(side_effect=ResponseExitForRecovery())), - ): - handler = cast( - AsyncGenerator[Any, None], - server._handle_response(request, context, cancellation_signal), # pyright: ignore[reportPrivateUsage] - ) - with pytest.raises(ResponseExitForRecovery): - _ = [event async for event in handler] - - # Only the restore-only replay call may have happened; the new-turn call must never fire. - assert inner.run_count <= run_count_after_first_turn + 1 - - async def test_previous_response_requires_existing_workflow_checkpoint(self) -> None: - """A previous_response_id naming a scope with no checkpoint must fail loudly rather than - silently starting a fresh workflow run (which could repeat side effects or misinterpret a - continuation as a new request).""" - workflow_agent, inner = _build_multi_update_workflow_agent(["hello"]) - server = _make_server(workflow_agent) - request = CreateResponse(model="m", input="hi", previous_response_id="response-missing") - context = ResponseContext(response_id="response-current", mode_flags=MagicMock()) - - with patch.object(ResponseContext, "get_input_items", new=AsyncMock(return_value=[])): - handler = server._handle_response(request, context, asyncio.Event()) # pyright: ignore[reportPrivateUsage] - events = [event async for event in handler] - - failed_events = [ - event for event in events if isinstance(event, Mapping) and event.get("type") == "response.failed" - ] - assert len(failed_events) == 1 - failed_event = cast(Mapping[str, Any], failed_events[0]) - response = cast(Mapping[str, Any], failed_event["response"]) - error = cast(Mapping[str, Any], response["error"]) - assert ( - "Cannot find an existing workflow checkpoint for previous_response_id=response-missing." in error["message"] - ) - assert inner.run_count == 0 - - async def test_non_streaming_emits_mcp_approval_request_and_persists_to_storage(self) -> None: - workflow_agent, mock_agent = _build_approval_workflow_agent(approval_request_id="apr_wf_ns") - server = _make_server(workflow_agent) - - resp = await _post(server, stream=False) - assert resp.status_code == 200 - body = resp.json() - assert body["status"] == "completed" - - approval_items = [it for it in body["output"] if it["type"] == "mcp_approval_request"] - assert len(approval_items) == 1 - assert approval_items[0]["name"] == "delete_file" - assert approval_items[0]["server_label"] == "test_server" - approval_request_id = approval_items[0]["id"] - - # The id surfaced over the wire is generated by the response stream - # builder; the original approval ``Content`` (carrying the inner - # ``function_call``) must be persisted under that id so the next - # turn can reconstruct it. - loaded = await server._function_approval_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] - config=server.config, platform_context=get_request_context() - ).load_approval_request(approval_request_id) - assert loaded.type == "function_approval_request" - assert loaded.function_call is not None - assert loaded.function_call.name == "delete_file" - assert mock_agent.run_count == 1 - - async def test_streaming_emits_mcp_approval_request_and_persists_to_storage(self) -> None: - workflow_agent, mock_agent = _build_approval_workflow_agent(approval_request_id="apr_wf_st") - server = _make_server(workflow_agent) - - resp = await _post(server, stream=True) - assert resp.status_code == 200 - - events = _parse_sse_events(resp.text) - types = _sse_event_types(events) - assert types[0] == "response.created" - assert types[-1] == "response.completed" - - approval_request_id: str | None = None - for e in events: - if e["event"] != "response.output_item.added": - continue - item: dict[str, Any] = e["data"].get("item") or {} - if item.get("type") == "mcp_approval_request": - approval_request_id = item.get("id") - break - assert approval_request_id is not None - - loaded = await server._function_approval_storage_provider.get_store( # pyright: ignore[reportPrivateUsage] - config=server.config, platform_context=get_request_context() - ).load_approval_request(approval_request_id) - assert loaded.type == "function_approval_request" - assert mock_agent.run_count == 1 - - async def test_round_trip_approval_response_resumes_workflow_agent(self) -> None: - """Two-turn HTTP round-trip: - - Turn 1 emits ``mcp_approval_request`` and writes a workflow - checkpoint under the response id. Turn 2 sends the - ``mcp_approval_response`` with ``previous_response_id`` set, so the - host restores the checkpoint, the WorkflowAgent routes the - approval response back to the paused inner agent, and the inner - agent emits the final assistant text. - """ - workflow_agent, mock_agent = _build_approval_workflow_agent( - approval_request_id="apr_wf_rt", - final_text="done with approval", - ) - server = _make_server(workflow_agent) - checkpoint_provider = server._checkpoint_storage_provider # pyright: ignore[reportPrivateUsage] - - with patch.object(checkpoint_provider, "get_store", wraps=checkpoint_provider.get_store) as get_store: - first = await _post(server, stream=False) - assert first.status_code == 200 - first_body = first.json() - first_response_id = first_body["id"] - approval_items = [it for it in first_body["output"] if it["type"] == "mcp_approval_request"] - assert len(approval_items) == 1 - approval_request_id = approval_items[0]["id"] - assert mock_agent.run_count == 1 - - second_payload: dict[str, Any] = { - "model": "test-model", - "input": [ - { - "type": "mcp_approval_response", - "approval_request_id": approval_request_id, - "approve": True, - } - ], - "stream": False, - "previous_response_id": first_response_id, - } - second = await _post_json(server, second_payload) - assert second.status_code == 200 - second_body = second.json() - assert second_body["status"] == "completed" - assert [call.kwargs["context_id"] for call in get_store.call_args_list] == [ - first_response_id, - second_body["id"], - first_response_id, - ] - - # The inner agent must have been resumed (restore replay + new turn). - # Restore call is a no-op for the mock (no input); the new-turn call - # delivers the approval response, so run_count grows by at least 1. - assert mock_agent.run_count >= 2 - - # The final assistant text from the resumed inner agent surfaces in - # the HTTP output. - text_pieces = [ - part.get("text", "") - for item in second_body["output"] - if item["type"] == "message" - for part in item.get("content", []) - if part.get("type") == "output_text" - ] - assert any("done with approval" in t for t in text_pieces), ( - f"expected resumed workflow output, got {second_body['output']}" - ) - - # The new-turn invocation of the inner agent must have received the - # approval response routed back through WorkflowAgent. - approval_responses = [ - c for m in mock_agent.last_run_messages for c in m.contents if c.type == "function_approval_response" - ] - assert len(approval_responses) == 1 - assert approval_responses[0].approved is True - - async def test_round_trip_approval_response_streaming(self) -> None: - """Streaming variant of the round-trip: turn 2 is requested with - ``stream=true`` and surfaces the resumed text as SSE events.""" - workflow_agent, mock_agent = _build_approval_workflow_agent( - approval_request_id="apr_wf_rt_st", - final_text="streamed-done", - ) - server = _make_server(workflow_agent) - - first = await _post(server, stream=False) - first_body = first.json() - first_response_id = first_body["id"] - approval_request_id = next(it["id"] for it in first_body["output"] if it["type"] == "mcp_approval_request") - - second = await _post_json( - server, - { - "model": "test-model", - "input": [ - { - "type": "mcp_approval_response", - "approval_request_id": approval_request_id, - "approve": True, - } - ], - "stream": True, - "previous_response_id": first_response_id, - }, - ) - assert second.status_code == 200 - events = _parse_sse_events(second.text) - types = _sse_event_types(events) - assert types[0] == "response.created" - assert types[-1] == "response.completed" - - text_done = [e for e in events if e["event"] == "response.output_text.done"] - assert any("streamed-done" in e["data"]["text"] for e in text_done) - assert mock_agent.run_count >= 2 - - async def test_round_trip_approval_response_rejected(self) -> None: - """Sending ``approve=False`` must surface as ``approved=False`` to the - inner agent on resume.""" - workflow_agent, mock_agent = _build_approval_workflow_agent( - approval_request_id="apr_wf_reject", - final_text="acknowledged", - ) - server = _make_server(workflow_agent) - - first = await _post(server, stream=False) - first_body = first.json() - first_response_id = first_body["id"] - approval_request_id = next(it["id"] for it in first_body["output"] if it["type"] == "mcp_approval_request") - - second = await _post_json( - server, - { - "model": "test-model", - "input": [ - { - "type": "mcp_approval_response", - "approval_request_id": approval_request_id, - "approve": False, - } - ], - "stream": False, - "previous_response_id": first_response_id, - }, - ) - assert second.status_code == 200 - - approval_responses = [ - c for m in mock_agent.last_run_messages for c in m.contents if c.type == "function_approval_response" - ] - assert len(approval_responses) == 1 - assert approval_responses[0].approved is False - - -# endregion - - -# region Resilient background checkpointing - - -class TestResilientBackgroundCheckpointing: - """``ResponseEventStream.checkpoint()`` only persists when its returned event is ``yield``-ed, so - ``_handle_inner_workflow`` fetches the latest saved workflow checkpoint and yields it after every update - when resilient_background is enabled. - """ - - async def test_workflow_yields_checkpoint_event_when_resilient_background(self, tmp_path: Path) -> None: - workflow_agent = _build_text_workflow_agent("hello from workflow") - server = _make_server( - workflow_agent, - response_store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), - ) - request = CreateResponse(model="m", input="hi", background=True, stream=True, store=True) - context = ResponseContext(response_id="response-current", mode_flags=MagicMock()) - - events = [ - event - async for event in server._handle_response( # pyright: ignore[reportPrivateUsage] - request, context, asyncio.Event() - ) - ] - - checkpoint_events = [e for e in events if isinstance(e, ResponseCheckpointEvent)] - assert checkpoint_events, "expected at least one checkpoint event yielded for a resilient background run" - - async def test_signalled_iterator_stamps_items_when_produced(self) -> None: - """The stamp reflects state as of production, not consumption. - - The driver runs one item ahead: once the consumer holds item k, the wrapped iterator may - already have resumed and created a checkpoint after it. The stamp taken right after item k - was produced must not see that later checkpoint. - """ - checkpoints: list[int] = [] - - async def produce() -> AsyncIterator[int]: - for k in range(1, 4): - yield k - # Runs when the iterator is resumed to produce the next item, i.e. after item k - # was handed over, mirroring the runner checkpointing at the end of a superstep. - checkpoints.append(k) - - async def stamp() -> int: - return len(checkpoints) - - seen: list[tuple[int, int, int]] = [] - it = _SignalledIterator(produce(), asyncio.Event(), stamp=stamp) - async with aclosing(it): - async for item in it: - # Give the driver every chance to run ahead before we look at the stamp. - for _ in range(5): - await asyncio.sleep(0) - seen.append((item, it.stamp, len(checkpoints))) - - assert [(item, stamped) for item, stamped, _ in seen] == [(1, 0), (2, 1), (3, 2)] - # Consumption-time state had already moved past the stamped one for every item. - assert all(consumed > stamped for _, stamped, consumed in seen) - - async def test_snapshots_pair_output_with_the_checkpoint_it_follows(self, tmp_path: Path) -> None: - """Every persisted snapshot must contain exactly the output emitted before it, and the final - snapshot must carry the full output and the incomplete reason. - """ - workflow_agent = _build_text_workflow_agent("filtered by workflow", finish_reason="content_filter") - server = _make_server( - workflow_agent, - response_store=FileResponseStore(storage_dir=tmp_path), - options=ResponsesServerOptions(resilient_background=True), - ) - request = CreateResponse(model="m", input="hi", background=True, stream=True, store=True) - context = ResponseContext(response_id="response-current", mode_flags=MagicMock()) - - emitted_text = "" - snapshots: list[tuple[str, dict[str, Any]]] = [] - async for event in server._handle_response( # pyright: ignore[reportPrivateUsage] - request, context, asyncio.Event() - ): - if isinstance(event, ResponseCheckpointEvent): - # The event references the live response; copy it as it is at persistence time. - snapshots.append((emitted_text, copy.deepcopy(dict(event.response)))) - elif isinstance(event, Mapping) and event.get("type") == "response.output_text.delta": - emitted_text += str(event.get("delta", "")) - - assert emitted_text == "filtered by workflow" - assert snapshots, "expected the completed workflow to be snapshotted" - checkpoint_ids: list[str] = [] - for text_before, response in snapshots: - internal = response["internal_metadata"] - checkpoint_ids.append(internal[_LATEST_CHECKPOINT_ID_KEY]) - snapshot_text = "".join( - part["text"] - for item in response["output"] - if item["type"] == "message" - for part in item["content"] - if part["type"] == "output_text" - ) - assert snapshot_text == text_before - assert len(set(checkpoint_ids)) == len(checkpoint_ids), "each checkpoint is snapshotted once" - - # The last snapshot is paired with the workflow's final checkpoint and carries everything. - final_text, final_response = snapshots[-1] - assert final_text == "filtered by workflow" - assert final_response["internal_metadata"][_INCOMPLETE_REASON_KEY] == "content_filter" - assert final_response["status"] == "in_progress" - - -# endregion - - -# region Parallel pre-model reads (_load_request_messages) - - -class TestParallelRequestReads: - """Covers the concurrent input/history read helper introduced to remove serial - latency from the request critical path (overlap, ordering, and no orphaned - storage reads when one side fails).""" - - @staticmethod - def _identity_converters(monkeypatch: pytest.MonkeyPatch) -> None: - async def _passthrough(items: Any, *, approval_storage: Any = None) -> list[Any]: - del approval_storage - return list(items) - - monkeypatch.setattr("agent_framework_foundry_hosting._responses._items_to_messages", _passthrough) - monkeypatch.setattr("agent_framework_foundry_hosting._responses._output_items_to_messages", _passthrough) - - async def test_reads_overlap_and_preserve_order(self, monkeypatch: pytest.MonkeyPatch) -> None: - self._identity_converters(monkeypatch) - server = _make_server(_make_agent()) - server._uses_agent_server_history = True # pyright: ignore[reportPrivateUsage] - - history_msg = Message(role="assistant", contents=[Content.from_text("H")]) - input_msg = Message(role="user", contents=[Content.from_text("I")]) - - input_started = asyncio.Event() - history_started = asyncio.Event() - release = asyncio.Event() - - async def get_input_items() -> list[Message]: - input_started.set() - await release.wait() - return [input_msg] - - async def get_history() -> list[Message]: - history_started.set() - await release.wait() - return [history_msg] - - context = MagicMock(spec=ResponseContext) - context.get_input_items = get_input_items - context.get_history = get_history - - task = asyncio.ensure_future(server._load_request_messages(context, approval_storage=None)) # pyright: ignore[reportPrivateUsage] - try: - # Both reads must be in-flight before either is allowed to finish — proves they overlap. - await asyncio.wait_for(input_started.wait(), timeout=1) - await asyncio.wait_for(history_started.wait(), timeout=1) - release.set() - messages = await asyncio.wait_for(task, timeout=1) - finally: - release.set() - - # History precedes input in the assembled model input, and the helper owns the ordering. - assert messages == [history_msg, input_msg] - - async def test_history_read_skipped_without_agent_server_history(self, monkeypatch: pytest.MonkeyPatch) -> None: - self._identity_converters(monkeypatch) - server = _make_server(_make_agent()) - server._uses_agent_server_history = False # pyright: ignore[reportPrivateUsage] - - input_msg = Message(role="user", contents=[Content.from_text("I")]) - - async def get_input_items() -> list[Message]: + async def get_input_items() -> list[Item]: return [input_msg] context = MagicMock(spec=ResponseContext) context.get_input_items = get_input_items context.get_history = AsyncMock() - messages = await server._load_request_messages(context, approval_storage=None) # pyright: ignore[reportPrivateUsage] + messages = await server._load_request_messages(context) # pyright: ignore[reportPrivateUsage] assert messages == [input_msg] context.get_history.assert_not_awaited() @@ -10679,7 +10390,7 @@ async def get_history() -> list[Message]: with pytest.raises(RuntimeError, match="input read boom"): await asyncio.wait_for( - server._load_request_messages(context, approval_storage=None), # pyright: ignore[reportPrivateUsage] + server._load_request_messages(context), # pyright: ignore[reportPrivateUsage] timeout=2, ) diff --git a/python/packages/foundry_hosting/tests/test_responses_int.py b/python/packages/foundry_hosting/tests/test_responses_int.py index edd4138bcc9..a84c571fa38 100644 --- a/python/packages/foundry_hosting/tests/test_responses_int.py +++ b/python/packages/foundry_hosting/tests/test_responses_int.py @@ -40,16 +40,22 @@ SlidingWindowStrategy, WorkflowBuilder, WorkflowContext, - executor, handler, tool, ) from agent_framework.foundry import FoundryChatClient, FoundryChatOptions -from azure.ai.agentserver.responses import InMemoryResponseProvider, ResponsesServerOptions +from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled +from azure.ai.agentserver.responses import InMemoryResponseProvider from azure.identity import AzureCliCredential from openai import AsyncOpenAI, DefaultAsyncHttpxClient -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ( + ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, + WorkflowTurn, + response_input_messages, +) _OPENAI_HTTPX = cast(Any, import_module(DefaultAsyncHttpxClient.__mro__[1].__module__.partition(".")[0])) @@ -78,7 +84,7 @@ def server() -> ResponsesHostServer: instructions="You are a concise assistant. Keep answers very short (one or two sentences).", ) - return ResponsesHostServer(agent, store=InMemoryResponseProvider()) + return ResponsesHostServer(agent, response_store=InMemoryResponseProvider()) @tool @@ -87,7 +93,7 @@ async def get_weather(location: Annotated[str, "The city name"]) -> str: return f"The weather in {location} is 72°F and sunny." -@pytest.fixture(params=["agent_server", "agent"], ids=["agent-server-history", "agent-managed-service-history"]) +@pytest.fixture(params=["host_managed", "agent_managed"], ids=["host-managed-history", "agent-managed-service-history"]) def history_server(request: pytest.FixtureRequest) -> ResponsesHostServer: """Create a real Foundry server for each model-history source.""" client = FoundryChatClient(credential=AzureCliCredential()) # pyrefly: ignore[bad-argument-type] @@ -98,12 +104,12 @@ def history_server(request: pytest.FixtureRequest) -> ResponsesHostServer: ) return ResponsesHostServer( agent, - store=InMemoryResponseProvider(), + response_store=InMemoryResponseProvider(), history_source=request.param, ) -@pytest.fixture(params=["agent_server", "agent"], ids=["agent-server-history", "agent-managed-service-history"]) +@pytest.fixture(params=["host_managed", "agent_managed"], ids=["host-managed-history", "agent-managed-service-history"]) def history_server_with_tools(request: pytest.FixtureRequest) -> ResponsesHostServer: """Create a real Foundry tool-calling server for each model-history source.""" client = FoundryChatClient(credential=AzureCliCredential()) # pyrefly: ignore[bad-argument-type] @@ -115,7 +121,7 @@ def history_server_with_tools(request: pytest.FixtureRequest) -> ResponsesHostSe ) return ResponsesHostServer( agent, - store=InMemoryResponseProvider(), + response_store=InMemoryResponseProvider(), history_source=request.param, ) @@ -595,8 +601,8 @@ async def test_agent_history_with_in_memory_provider(self) -> None: ) server = ResponsesHostServer( agent, - store=InMemoryResponseProvider(), - history_source="agent", + response_store=InMemoryResponseProvider(), + history_source="agent_managed", ) first = await _post_json( @@ -765,7 +771,7 @@ async def foundry_responses_boundary(request: Any) -> Any: "include": ["reasoning.encrypted_content"], }, ) - server = ResponsesHostServer(agent, store=InMemoryResponseProvider()) + server = ResponsesHostServer(agent, response_store=InMemoryResponseProvider()) first = await _post_json( server, @@ -901,7 +907,7 @@ async def test_temperature_and_max_tokens(self, server: ResponsesHostServer) -> # --------------------------------------------------------------------------- # Tests — real crash/recovery for resilient-background workflows # -# A real ResponsesHostServer is force-killed mid-workflow, then a freshly +# A real ResponsesWorkflowHostServer is force-killed mid-workflow, then a freshly # started process pointed at the same on-disk state recovers and completes the same response. # The workflow is deterministic and model-free so the test needs no credentials. # --------------------------------------------------------------------------- @@ -989,22 +995,26 @@ async def call_tool(self, target: int, ctx: WorkflowContext[str, Content | str]) await ctx.send_message("Countdown complete.", target_id="complete") -@executor(id="complete") -async def _countdown_complete(message: str, ctx: WorkflowContext[Never, str]) -> None: # zuban: ignore - """Yield the workflow's completion output.""" - await ctx.yield_output(message) +class _CountdownCompleteExecutor(Executor): + def __init__(self) -> None: + super().__init__(id="complete") + + @handler + async def complete(self, message: str, ctx: WorkflowContext[Never, str]) -> None: + await ctx.yield_output(message) def _build_countdown_workflow(sleep_seconds: float): """Build the target extraction, countdown, and completion workflow.""" start = _CountdownStartExecutor() countdown = _CountdownExecutor(sleep_seconds) + complete = _CountdownCompleteExecutor() return ( - WorkflowBuilder(start_executor=start, output_from="all") + WorkflowBuilder(name="countdown-workflow", start_executor=start, output_from="all") .add_edge(start, countdown) .add_edge(countdown, countdown) - .add_edge(countdown, _countdown_complete) + .add_edge(countdown, complete) .build() ) @@ -1013,12 +1023,13 @@ def _build_paired_yield_workflow(sleep_seconds: float): """Build a workflow whose self-looping executor yields two output items per superstep.""" start = _CountdownStartExecutor() paired = _PairedYieldExecutor(sleep_seconds) + complete = _CountdownCompleteExecutor() return ( - WorkflowBuilder(start_executor=start, output_from="all") + WorkflowBuilder(name="paired-yield-workflow", start_executor=start, output_from="all") .add_edge(start, paired) .add_edge(paired, paired) - .add_edge(paired, _countdown_complete) + .add_edge(paired, complete) .build() ) @@ -1027,11 +1038,12 @@ def _build_tool_call_workflow(sleep_seconds: float): """Build a workflow that emits a function-call/result pair, then text, then a second superstep.""" start = _CountdownStartExecutor() tool_call = _ToolCallExecutor(sleep_seconds) + complete = _CountdownCompleteExecutor() return ( - WorkflowBuilder(start_executor=start, output_from="all") + WorkflowBuilder(name="tool-call-workflow", start_executor=start, output_from="all") .add_edge(start, tool_call) - .add_edge(tool_call, _countdown_complete) + .add_edge(tool_call, complete) .build() ) @@ -1049,9 +1061,16 @@ def _run_resilient_server( os.dup2(log_file.fileno(), 1) os.dup2(log_file.fileno(), 2) os.environ["AGENTSERVER_STATE_ROOT"] = state_root + set_resilient_tasks_enabled(True) - workflow_agent = build_workflow(sleep_seconds).as_agent(name="resilient-workflow") - server = ResponsesHostServer(workflow_agent, options=ResponsesServerOptions(resilient_background=True)) + async def parse_input(request: ResponsesRequest) -> WorkflowTurn[list[Message]]: + return WorkflowTurn(input=await response_input_messages(request)) + + server = ResponsesWorkflowHostServer( + workflow=lambda request: build_workflow(sleep_seconds), + parse_input=parse_input, + resilient_background=True, + ) server.run(host="127.0.0.1", port=port) @@ -1226,7 +1245,7 @@ async def _crash_and_recover( strict=False, ) class TestWorkflowResilientRecoveryRealCrash: - """Force-kill a real ResponsesHostServer process mid-workflow and verify a freshly started + """Force-kill a real ResponsesWorkflowHostServer process mid-workflow and verify a freshly started process, pointed at the same on-disk state, recovers and completes the response with no lost or duplicated output. diff --git a/python/packages/foundry_hosting/tests/test_state_store.py b/python/packages/foundry_hosting/tests/test_state_store.py index 5c622ca1e75..e80411ca77f 100644 --- a/python/packages/foundry_hosting/tests/test_state_store.py +++ b/python/packages/foundry_hosting/tests/test_state_store.py @@ -6,19 +6,17 @@ from unittest.mock import AsyncMock, MagicMock, patch import pytest -from agent_framework import AgentSession, Content, WorkflowCheckpoint, WorkflowCheckpointException +from agent_framework import AgentSession, WorkflowCheckpoint, WorkflowCheckpointException from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext -from azure.ai.agentserver.core.storage import FoundryStateStore, FoundryStorageConflictError +from azure.ai.agentserver.core.storage import FoundryStateStore from agent_framework_foundry_hosting import ContextScopedStoreProvider, StoreProvider -from agent_framework_foundry_hosting._state_store import ( +from agent_framework_foundry_hosting._shared import ( AgentSessionStoreProvider, CheckpointStoreProvider, FoundryAgentSessionStore, FoundryCheckpointStore, - FoundryFunctionApprovalStore, - FunctionApprovalStoreProvider, ) @@ -82,7 +80,6 @@ def _platform_context( def test_storage_providers_use_public_abstraction() -> None: assert issubclass(CheckpointStoreProvider, ContextScopedStoreProvider) assert not issubclass(CheckpointStoreProvider, StoreProvider) - assert issubclass(FunctionApprovalStoreProvider, StoreProvider) assert issubclass(AgentSessionStoreProvider, StoreProvider) @@ -91,7 +88,7 @@ async def test_save_uses_context_scoped_store() -> None: checkpoint = _checkpoint("checkpoint-1") with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ) as get_or_create: result = await FoundryCheckpointStore("context-1", _platform_context()).save(checkpoint) @@ -107,7 +104,7 @@ async def test_load_returns_checkpoint() -> None: store.get_item = AsyncMock(return_value=SimpleNamespace(value=checkpoint.to_dict())) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): result = await FoundryCheckpointStore("context-1", _platform_context()).load("checkpoint-1") @@ -130,7 +127,7 @@ async def test_load_restricts_checkpoint_deserialization() -> None: with ( patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ), pytest.raises(WorkflowCheckpointException), @@ -147,7 +144,7 @@ async def test_load_accepts_a_declared_checkpoint_type() -> None: store.get_item = AsyncMock(return_value=SimpleNamespace(value=value)) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): result = await FoundryCheckpointStore( @@ -171,7 +168,7 @@ async def test_list_checkpoints_restricts_checkpoint_deserialization() -> None: with ( patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ), pytest.raises(WorkflowCheckpointException), @@ -201,7 +198,7 @@ async def test_provider_forwards_allowed_checkpoint_types() -> None: ) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): result = await storage.load("checkpoint-1") @@ -225,7 +222,7 @@ async def test_provider_restricts_by_default() -> None: with ( patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ), pytest.raises(WorkflowCheckpointException), @@ -239,7 +236,7 @@ async def test_load_raises_for_missing_checkpoint() -> None: with ( patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ), pytest.raises(WorkflowCheckpointException, match="No checkpoint found with ID missing"), @@ -268,7 +265,7 @@ async def test_list_checkpoints_paginates_and_filters_by_workflow() -> None: ) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): result = await FoundryCheckpointStore("context-1", _platform_context()).list_checkpoints( @@ -287,7 +284,7 @@ async def test_delete_reports_whether_checkpoint_existed(deleted_id: str | None, store.delete_item = AsyncMock(return_value=SimpleNamespace(id=deleted_id)) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): result = await FoundryCheckpointStore("context-1", _platform_context()).delete("checkpoint-1") @@ -387,102 +384,13 @@ def test_checkpoint_stores_require_context_id(create_store: Callable[[], Any]) - create_store() -def _approval_request(approval_request_id: str) -> Content: - function_call = Content.from_function_call( - "call-1", - "delete_file", - arguments='{"path": "/foo"}', - additional_properties={"server_label": "my_server"}, - ) - return Content.from_function_approval_request(approval_request_id, function_call) - - -async def test_save_and_load_function_approval_request() -> None: - store = _store() - request = _approval_request("approval-1") - store.get_item = AsyncMock(return_value=SimpleNamespace(value=request.to_dict())) - - with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", - new=AsyncMock(return_value=store), - ) as get_or_create: - storage = FoundryFunctionApprovalStore(_platform_context()) - await storage.save_approval_request("approval-1", request) - loaded = await storage.load_approval_request("approval-1") - - assert get_or_create.await_count == 2 - get_or_create.assert_awaited_with("function_approvals", user_isolation=True) - store.create_item.assert_awaited_once_with("approval-1", request.to_dict(), call_id="call-1") - store.get_item.assert_awaited_once_with("approval-1", call_id="call-1") - assert loaded == request - function_call = loaded.function_call - assert function_call is not None - assert function_call.name == "delete_file" - assert function_call.additional_properties["server_label"] == "my_server" - - -async def test_save_duplicate_function_approval_request_raises() -> None: - store = _store() - store.create_item = AsyncMock(side_effect=FoundryStorageConflictError("already exists")) - - with ( - patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", - new=AsyncMock(return_value=store), - ), - pytest.raises(ValueError, match="Approval request with ID 'approval-1' already exists"), - ): - await FoundryFunctionApprovalStore(_platform_context()).save_approval_request( - "approval-1", _approval_request("approval-1") - ) - - -async def test_load_missing_function_approval_request_raises() -> None: - store = _store() - store.get_item = AsyncMock(return_value=None) - - with ( - patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", - new=AsyncMock(return_value=store), - ), - pytest.raises(KeyError, match="Approval request with ID 'missing' does not exist"), - ): - await FoundryFunctionApprovalStore(_platform_context()).load_approval_request("missing") - - -@pytest.mark.parametrize("is_hosted", [True, False]) -def test_function_approval_storage_provider_uses_foundry_store(is_hosted: bool) -> None: - provider = FunctionApprovalStoreProvider() - config = _config(is_hosted=is_hosted) - platform_context = _platform_context() - - storage = provider.get_store(config=config, platform_context=platform_context) - - assert type(storage) is FoundryFunctionApprovalStore - assert storage.platform_context is platform_context - - -def test_function_approval_storage_provider_creates_request_scoped_storage() -> None: - with patch("agent_framework_foundry_hosting._state_store.FoundryFunctionApprovalStore") as storage_type: - provider = FunctionApprovalStoreProvider() - config = _config(is_hosted=False) - first_context = _platform_context("call-1") - second_context = _platform_context("call-2") - provider.get_store(config=config, platform_context=first_context) - provider.get_store(config=config, platform_context=second_context) - - assert storage_type.call_args_list[0].args == (first_context,) - assert storage_type.call_args_list[1].args == (second_context,) - - async def test_set_agent_session_uses_scoped_store() -> None: store = _store() session = AgentSession(session_id="agent-session-1") session.state["turn_count"] = 2 with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ) as get_or_create: await FoundryAgentSessionStore(_platform_context()).set("storage-session-1", session) @@ -494,7 +402,7 @@ async def test_set_agent_session_uses_scoped_store() -> None: async def test_agent_session_provider_selects_a_separate_logical_store() -> None: store = _store() with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ) as get_or_create: provider = AgentSessionStoreProvider(store_name="invocation_sessions") @@ -551,7 +459,7 @@ async def test_get_agent_session_returns_deserialized_session() -> None: store.get_item = AsyncMock(return_value=SimpleNamespace(value=session.to_dict(), etag="etag-1")) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ) as get_or_create: result = await FoundryAgentSessionStore(_platform_context()).get("storage-session-1") @@ -568,7 +476,7 @@ async def test_get_missing_agent_session_returns_none() -> None: store.get_item = AsyncMock(return_value=None) with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): result = await FoundryAgentSessionStore(_platform_context()).get("missing") @@ -580,7 +488,7 @@ async def test_delete_agent_session_is_idempotent() -> None: store = _store() with patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ): await FoundryAgentSessionStore(_platform_context()).delete("storage-session-1") @@ -619,7 +527,7 @@ def test_agent_session_storage_provider_uses_foundry_store(is_hosted: bool) -> N def test_agent_session_storage_provider_creates_request_scoped_storage() -> None: - with patch("agent_framework_foundry_hosting._state_store.FoundryAgentSessionStore") as storage_type: + with patch("agent_framework_foundry_hosting._shared.FoundryAgentSessionStore") as storage_type: provider = AgentSessionStoreProvider() config = _config(is_hosted=False) first_context = _platform_context("call-1") @@ -631,7 +539,7 @@ def test_agent_session_storage_provider_creates_request_scoped_storage() -> None assert storage_type.call_args_list[1].args == (second_context,) -@pytest.mark.parametrize("provider", ["agent", "checkpoint", "approval"]) +@pytest.mark.parametrize("provider", ["agent", "checkpoint"]) @pytest.mark.parametrize( ("platform_session_id", "request_session_id", "user_id", "call_id"), [ @@ -649,10 +557,8 @@ def test_hosted_store_providers_reject_missing_platform_identity( with pytest.raises(RuntimeError, match="FOUNDRY_AGENT_SESSION_ID|does not match|user ID and call ID"): if provider == "agent": AgentSessionStoreProvider().get_store(config=config, platform_context=context) - elif provider == "checkpoint": - CheckpointStoreProvider().get_store(config=config, context_id="workflow", platform_context=context) else: - FunctionApprovalStoreProvider().get_store(config=config, platform_context=context) + CheckpointStoreProvider().get_store(config=config, context_id="workflow", platform_context=context) async def test_hosted_store_names_partition_sandbox_and_forward_call_id() -> None: @@ -669,7 +575,7 @@ async def get_store(name: str, *, user_isolation: bool) -> MagicMock: _platform_context(call_id="call-2", session_id="other-sandbox"), ] - with patch("agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", new=get_store): + with patch("agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=get_store): for context in contexts: config = _config(is_hosted=True, session_id=context.session_id) sessions = AgentSessionStoreProvider().get_store(config=config, platform_context=context) @@ -678,10 +584,8 @@ async def get_store(name: str, *, user_isolation: bool) -> MagicMock: config=config, context_id="../../conversation", platform_context=context ) await checkpoints.save(_checkpoint("checkpoint-1")) - approvals = FunctionApprovalStoreProvider().get_store(config=config, platform_context=context) - await approvals.save_approval_request("approval-1", _approval_request("approval-1")) - assert len(names) == len(set(names)) == 6 + assert len(names) == len(set(names)) == 4 assert all(name.split("/")[1] == "v2" and len(name) <= 128 for name in names) assert all("user-1" not in name and "sandbox" not in name and "conversation" not in name for name in names) assert {call.kwargs["call_id"] for call in store.create_item.await_args_list} == {"call-1", "call-2"} @@ -704,38 +608,29 @@ async def test_hosted_scope_does_not_read_existing_unscoped_agent_session() -> N assert await legacy.get_item("response-1", call_id=context.call_id) is not None -async def test_hosted_scope_does_not_read_existing_unscoped_checkpoint_or_approval() -> None: +async def test_hosted_scope_does_not_read_existing_unscoped_checkpoint() -> None: context = _platform_context() legacy_checkpoint = await FoundryStateStore.get_or_create("checkpoints/conversation-1", user_isolation=True) - legacy_approval = await FoundryStateStore.get_or_create("function_approvals", user_isolation=True) async with legacy_checkpoint: await legacy_checkpoint.create_item( "checkpoint-1", encode_checkpoint_value(_checkpoint("checkpoint-1").to_dict()), call_id=context.call_id ) - async with legacy_approval: - await legacy_approval.create_item( - "approval-1", _approval_request("approval-1").to_dict(), call_id=context.call_id - ) config = _config(is_hosted=True) checkpoints = CheckpointStoreProvider().get_store( config=config, context_id="conversation-1", platform_context=context ) - approvals = FunctionApprovalStoreProvider().get_store(config=config, platform_context=context) with pytest.raises(WorkflowCheckpointException, match="No checkpoint found"): await checkpoints.load("checkpoint-1") - with pytest.raises(KeyError, match="does not exist"): - await approvals.load_approval_request("approval-1") -async def test_hosted_session_checkpoint_and_approval_data_stay_in_one_sandbox() -> None: +async def test_hosted_session_and_checkpoint_data_stay_in_one_sandbox() -> None: config = _config(is_hosted=True) first_context = _platform_context(session_id="sandbox-1") second_context = _platform_context(session_id="sandbox-2") second_config = _config(is_hosted=True, session_id="sandbox-2") sessions = AgentSessionStoreProvider() checkpoints = CheckpointStoreProvider() - approvals = FunctionApprovalStoreProvider() first_session_store = sessions.get_store(config=config, platform_context=first_context) await first_session_store.set("response-1", AgentSession()) @@ -759,15 +654,6 @@ async def test_hosted_session_checkpoint_and_approval_data_stay_in_one_sandbox() with pytest.raises(WorkflowCheckpointException, match="No checkpoint found"): await second_checkpoint_store.load("checkpoint-1") - first_approval_store = approvals.get_store(config=config, platform_context=first_context) - await first_approval_store.save_approval_request("approval-1", _approval_request("approval-1")) - continued_approval_store = approvals.get_store(config=config, platform_context=continued_context) - assert await continued_approval_store.load_approval_request("approval-1") == _approval_request("approval-1") - second_approval_store = approvals.get_store(config=second_config, platform_context=second_context) - with pytest.raises(KeyError, match="does not exist"): - await second_approval_store.load_approval_request("approval-1") - assert await first_approval_store.load_approval_request("approval-1") == _approval_request("approval-1") - async def test_hosted_session_rejects_a_stale_etag_without_overwriting_the_winner() -> None: config = _config(is_hosted=True) @@ -833,7 +719,7 @@ async def test_loaded_session_without_an_etag_fails_closed() -> None: ) with ( patch( - "agent_framework_foundry_hosting._state_store.FoundryStateStore.get_or_create", + "agent_framework_foundry_hosting._shared.FoundryStateStore.get_or_create", new=AsyncMock(return_value=store), ), pytest.raises(RuntimeError, match="without an ETag"), diff --git a/python/packages/foundry_hosting/tests/test_toolbox.py b/python/packages/foundry_hosting/tests/test_toolbox.py index 6396df53df1..6dc701fc5bd 100644 --- a/python/packages/foundry_hosting/tests/test_toolbox.py +++ b/python/packages/foundry_hosting/tests/test_toolbox.py @@ -10,7 +10,7 @@ from collections.abc import Awaitable, Callable from datetime import datetime, timedelta, timezone from types import SimpleNamespace -from typing import Any, cast +from typing import Any, Literal, cast from unittest.mock import AsyncMock, Mock, patch import httpx as host_httpx @@ -18,6 +18,7 @@ import pytest from agent_framework import ( Agent, + AgentSession, FunctionInvocationContext, MCPStreamableHTTPTool, Message, @@ -31,6 +32,7 @@ reset_request_context, set_request_context, ) +from azure.ai.agentserver.responses import InMemoryResponseProvider from mcp.client.session import ClientSession from mcp.types import Prompt, Tool from pydantic import AnyUrl @@ -45,8 +47,16 @@ ) from .foundry_hosting.toolbox_approval_helpers import ( # pyrefly: ignore[missing-import] - pytest's importlib namespace + CONSENT_LINK, + ConsentCallerHeaders, + ConsentModelServer, + ConsentPendingInput, + ConsentSessionStoreProvider, ToolboxApprovalClient, ToolboxApprovalServer, + ToolboxConsentServer, + consent_model_client, + post_consent_response, toolbox_connection, toolbox_tool, ) @@ -1639,3 +1649,474 @@ async def test_caller_can_wrap_the_foundry_metadata_policy() -> None: caller_policy.return_value = "always_require" await toolbox.load_tools() assert toolbox.functions[0].approval_mode == "always_require" + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +@pytest.mark.parametrize("continuation", ["response", "conversation"]) +async def test_toolbox_oauth_consent_pairs_results_and_continues( + stream: bool, + history_source: Literal["host_managed", "inference_service"], + continuation: str, +) -> None: + mcp = ToolboxConsentServer() + model = ConsentModelServer() + sessions = ConsentSessionStoreProvider() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source=history_source, + response_store=InMemoryResponseProvider(), + agent_session_store_provider=sessions, + ) + payload: dict[str, Any] = {"input": "Run the tools", "stream": stream, "store": True} + if continuation == "conversation": + payload["conversation"] = "oauth-conversation" + first, events = await post_consent_response(host, payload) + + assert first["status"] == "incomplete", first + assert [item["consent_link"] for item in first["output"] if item["type"] == "oauth_consent_request"] == [ + CONSENT_LINK + ] + calls = [item for item in first["output"] if item["type"] == "function_call"] + results = [item for item in first["output"] if item["type"] == "function_call_output"] + assert len(calls) == len(results) == 3, first + assert [item["call_id"] for item in results] == [item["call_id"] for item in calls] + assert results[0]["output"] == "search-result" + assert "OAuth consent is required" in results[1]["output"] + assert results[2]["output"] == "search-result" + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected", "learn___later"] + assert "hidden settlement reply" not in json.dumps(first["output"]) + if stream: + assert ( + sum( + event["type"] == "response.output_item.done" and event["item"]["type"] == "oauth_consent_request" + for event in events + ) + == 1 + ) + saved = await sessions.store.get(first["id"]) + assert saved is not None + if history_source == "inference_service": + assert len(model.requests) == 2 + assert model.requests[1]["tool_choice"] == "none" + assert [item["type"] for item in model.requests[1]["input"]] == ["function_call_output"] * 3 + assert model.requests[1]["input"][0]["output"] == "search-result" + assert model.requests[1]["input"][2]["output"] == "search-result" + assert saved.service_session_id == "resp_model_2" + assert first["usage"]["input_tokens"] == 10 + assert first["usage"]["output_tokens"] == 4 + else: + assert len(model.requests) == 1 + assert saved.service_session_id is None + + mcp.authorized = True + payload["input"] = "I have signed in. Continue." + if continuation == "response": + payload["previous_response_id"] = first["id"] + second, _ = await post_consent_response(host, payload) + assert second["status"] == "completed", second + assert not any(item["type"] == "oauth_consent_request" for item in second["output"]) + assert [call["name"] for call in mcp.calls] == [ + "learn___completed", + "learn___protected", + "learn___later", + "learn___protected", + ] + assert not model.history_errors + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +@pytest.mark.parametrize("pending_kind", ["input", "approval"]) +@pytest.mark.parametrize("approval_resume", [False, True], ids=["ordinary", "approved-replay"]) +async def test_toolbox_oauth_consent_preserves_a_mixed_pending_batch( + stream: bool, + history_source: Literal["host_managed", "inference_service"], + pending_kind: Literal["input", "approval"], + approval_resume: bool, +) -> None: + mcp = ToolboxConsentServer() + mcp.tools = [ + toolbox_tool("always" if approval_resume else "never", name=name) + for name in ("learn___completed", "learn___protected") + ] + pending = ConsentPendingInput(pending_kind, require_approval=approval_resume) + names = ("learn___completed", "learn___protected", pending.function.name) + if pending_kind == "approval": + names = (pending.function.name, "learn___completed", "learn___protected") + model = ConsentModelServer(names) + sessions = ConsentSessionStoreProvider() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model, concurrent=True) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=[toolbox, pending.function], middleware=[pending]), + history_source=history_source, + response_store=InMemoryResponseProvider(), + agent_session_store_provider=sessions, + ) + first, _ = await post_consent_response(host, {"input": "Run the tools", "stream": stream}) + if approval_resume: + approvals = [item for item in first["output"] if item["type"] == "mcp_approval_request"] + assert len(approvals) == 3 + assert not mcp.calls and not pending.executions + first, _ = await post_consent_response( + host, + { + "input": [ + { + "type": "message", + "role": "user", + "content": [{"type": "input_text", "text": "Keep this instruction until input arrives."}], + }, + *[ + {"type": "mcp_approval_response", "approval_request_id": approval["id"], "approve": True} + for approval in approvals + ], + ], + "previous_response_id": first["id"], + "stream": stream, + }, + ) + assert first["status"] == "incomplete", first + assert [item["consent_link"] for item in first["output"] if item["type"] == "oauth_consent_request"] == [ + CONSENT_LINK + ] + results = [item for item in first["output"] if item["type"] == "function_call_output"] + expected_call_ids = [f"call_initial_{position}" for position in range(3)] + pending_call_id = expected_call_ids[names.index(pending.function.name)] + assert [item["call_id"] for item in results] == [ + call_id for call_id in expected_call_ids if call_id != pending_call_id + ] + assert results[0]["output"] == "search-result" + assert "OAuth consent is required" in results[1]["output"] + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected"] + assert len(model.requests) == 1 + assert pending.executions == (1 if pending_kind == "input" else 0) + if pending_kind == "approval": + approval = next(item for item in first["output"] if item["type"] == "mcp_approval_request") + decision = {"type": "mcp_approval_response", "approval_request_id": approval["id"], "approve": True} + else: + assert "Please supply the missing input." in json.dumps(first["output"]) + decision = {"type": "function_call_output", "call_id": pending_call_id, "output": "provided-input"} + + saved = await sessions.store.get(first["id"]) + assert saved is not None + await sessions.store.set(first["id"], AgentSession.from_dict(saved.to_dict())) + second, _ = await post_consent_response( + host, {"input": [decision], "previous_response_id": first["id"], "stream": stream} + ) + assert second["status"] == "completed", second + assert len(model.requests) == 2 + outputs = [item for item in model.requests[1]["input"] if item["type"] == "function_call_output"] + if history_source == "inference_service": + assert [item["call_id"] for item in outputs] == expected_call_ids + else: + assert len(outputs) == 3 + assert {item["call_id"]: item["output"] for item in outputs} == { + **{item["call_id"]: item["output"] for item in results}, + pending_call_id: "pending-result" if pending_kind == "approval" else "provided-input", + } + assert not model.history_errors + if approval_resume: + assert "Keep this instruction until input arrives." in json.dumps(model.requests[1]["input"]) + assert pending.executions == 1 + assert not any(item["type"] == "oauth_consent_request" for item in second["output"]) + + third, _ = await post_consent_response( + host, {"input": "Continue.", "previous_response_id": second["id"], "stream": stream} + ) + assert third["status"] == "completed", third + assert len(model.requests) == 3 + if history_source == "inference_service": + assert not any(item["type"] == "function_call_output" for item in model.requests[2]["input"]) + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected"] + assert pending.executions == 1 + assert not model.history_errors + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +async def test_toolbox_oauth_consent_rejects_unstored_requests( + stream: bool, history_source: Literal["host_managed", "inference_service"] +) -> None: + mcp = ToolboxConsentServer() + model = ConsentModelServer() + sessions = ConsentSessionStoreProvider() + provider = InMemoryResponseProvider() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source=history_source, + response_store=provider, + agent_session_store_provider=sessions, + ) + response, _ = await post_consent_response(host, {"input": "Run the tools", "stream": stream, "store": False}) + assert response["status"] == "failed", response + assert "requires store=true" in response["error"]["message"] + assert not any(item["type"] == "oauth_consent_request" for item in response["output"]) + assert await sessions.store.get(response["id"]) is None + assert not model.chains + async with host_httpx.AsyncClient( + transport=host_httpx.ASGITransport(app=host), base_url="http://test" + ) as client: + stored = await client.get(f"/responses/{response['id']}") + assert stored.status_code == 404 + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("unexpected_call", [False, True], ids=["refused", "unexpected-call"]) +async def test_toolbox_oauth_consent_reconciliation_failure_is_not_an_incomplete_response( + stream: bool, unexpected_call: bool +) -> None: + mcp = ToolboxConsentServer() + model = ConsentModelServer() + model.fail_settlement = not unexpected_call + model.unexpected_settlement_call = unexpected_call + sessions = ConsentSessionStoreProvider() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source="inference_service", + response_store=InMemoryResponseProvider(), + agent_session_store_provider=sessions, + ) + response, events = await post_consent_response(host, {"input": "Run the tools", "stream": stream}) + assert response["status"] == "failed", response + assert not any(item["type"] == "oauth_consent_request" for item in response["output"]) + assert not any(event.get("item", {}).get("type") == "oauth_consent_request" for event in events) + assert len(model.requests) == 2 + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected", "learn___later"] + if unexpected_call: + saved = await sessions.store.get(response["id"]) + assert saved is not None + assert saved.service_session_id == "resp_model_2" + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +async def test_toolbox_oauth_consent_allows_an_internally_queued_mcp_dispatch( + stream: bool, history_source: Literal["host_managed", "inference_service"] +) -> None: + mcp = ToolboxConsentServer() + model = ConsentModelServer(("learn___protected", "learn___later")) + headers = ConsentCallerHeaders() + mcp.before_consent = headers.later_queued.wait + async with ( + toolbox_connection(mcp, header_provider=headers.headers) as toolbox, + consent_model_client(model, concurrent=True) as chat_client, + ): + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox, middleware=[headers]), + response_store=InMemoryResponseProvider(), + history_source=history_source, + ) + response, _ = await post_consent_response(host, {"input": "Run the tools", "stream": stream}) + assert response["status"] == "incomplete", response + assert headers.later_queued.is_set() + assert [call["name"] for call in mcp.calls] == ["learn___protected", "learn___later"] + results = [item for item in response["output"] if item["type"] == "function_call_output"] + assert len(results) == 2 + assert results[1]["output"] == "search-result" + + +@pytest.mark.parametrize( + "error", + [ + {"code": -32000, "message": "ordinary tool failure"}, + {"code": -32006, "message": "https://consent.example.com/unstructured"}, + {"code": -32006, "message": '{"errors": {}}'}, + {"code": -32006, "message": '{"errors": [null, 1, {"type": "mcp", "error": null}]}'}, + {"code": -32006, "message": '{"errors": [{"type": "mcp", "error": {"code": "OTHER"}}]}'}, + { + "code": -32006, + "message": json.dumps({ + "errors": [{"type": "mcp", "error": {"code": "CONSENT_REQUIRED", "message": "javascript:x"}}] + }), + }, + ], +) +async def test_toolbox_oauth_consent_does_not_classify_unvalidated_errors(error: dict[str, Any]) -> None: + mcp = ToolboxConsentServer() + mcp.error = error + model = ConsentModelServer() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer(Agent(client=chat_client, tools=toolbox), response_store=InMemoryResponseProvider()) + response, _ = await post_consent_response(host, {"input": "Run the tools"}) + assert response["status"] == "completed", response + assert not any(item["type"] == "oauth_consent_request" for item in response["output"]) + assert len(mcp.calls) == 3 + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +async def test_toolbox_mixed_consent_errors_remain_function_failures( + stream: bool, history_source: Literal["host_managed", "inference_service"] +) -> None: + mcp = ToolboxConsentServer() + mcp.error["message"] = "tools/call failed " + json.dumps({ + "errors": [ + {"name": "learn", "type": "mcp", "error": {"code": "CONSENT_REQUIRED", "message": CONSENT_LINK}}, + {"name": "calendar", "type": "mcp", "error": {"code": "INTERNAL_ERROR", "message": "Upstream failure"}}, + ] + }) + model = ConsentModelServer() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source=history_source, + response_store=InMemoryResponseProvider(), + ) + response, events = await post_consent_response(host, {"input": "Run the tools", "stream": stream}) + + assert response["status"] == "completed", response + assert not any(item["type"] == "oauth_consent_request" for item in response["output"]) + assert not any(event.get("item", {}).get("type") == "oauth_consent_request" for event in events) + results = [item for item in response["output"] if item["type"] == "function_call_output"] + assert len(results) == 3 + assert results[0]["output"] == results[2]["output"] == "search-result" + assert results[1]["output"].startswith("Error:") + assert "OAuth consent is required" not in results[1]["output"] + assert CONSENT_LINK not in json.dumps(response["output"]) + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected", "learn___later"] + assert len(model.requests) == 2 + assert model.requests[1].get("tool_choice") != "none" + assert not model.history_errors + + +async def test_toolbox_oauth_consent_origin_allowlist_is_enforced() -> None: + mcp = ToolboxConsentServer() + model = ConsentModelServer() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + response_store=InMemoryResponseProvider(), + allowed_oauth_consent_origins=["https://other.example.com"], + ) + response, _ = await post_consent_response(host, {"input": "Run the tools"}) + assert response["status"] == "failed", response + assert not any(item["type"] == "oauth_consent_request" for item in response["output"]) + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +async def test_toolbox_oauth_consent_remains_distinct_from_native_policy_approval( + stream: bool, history_source: Literal["host_managed", "inference_service"] +) -> None: + mcp = ToolboxConsentServer() + mcp.tools = [ + toolbox_tool("always", name=name) for name in ("learn___completed", "learn___protected", "learn___later") + ] + model = ConsentModelServer() + sessions = ConsentSessionStoreProvider() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source=history_source, + response_store=InMemoryResponseProvider(), + agent_session_store_provider=sessions, + ) + first, _ = await post_consent_response(host, {"input": "Run the tools", "stream": stream}) + approvals = [item for item in first["output"] if item["type"] == "mcp_approval_request"] + assert len(approvals) == 3 + assert not mcp.calls + assert not any(item["type"] == "oauth_consent_request" for item in first["output"]) + second, _ = await post_consent_response( + host, + { + "input": [ + { + "type": "message", + "role": "user", + "content": [{"type": "input_text", "text": "Keep this instruction."}], + }, + *[ + {"type": "mcp_approval_response", "approval_request_id": approval["id"], "approve": True} + for approval in approvals + ], + ], + "previous_response_id": first["id"], + "stream": stream, + }, + ) + assert second["status"] == "incomplete", second + assert [item["consent_link"] for item in second["output"] if item["type"] == "oauth_consent_request"] == [ + CONSENT_LINK + ] + results = [item for item in second["output"] if item["type"] == "function_call_output"] + assert len(results) == 3 + assert results[0]["output"] == "search-result" + assert results[2]["output"] == "search-result" + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected", "learn___later"] + assert not model.history_errors + saved = await sessions.store.get(second["id"]) + assert saved is not None + if history_source == "inference_service": + assert saved.service_session_id == "resp_model_2" + assert len(model.requests) == 2 + provider_input = model.requests[1]["input"] + assert sum(item["type"] == "function_call_output" for item in provider_input) == 3 + assert not any(item["type"] in ("function_call", "mcp_approval_response") for item in provider_input) + assert any("Keep this instruction." in json.dumps(item) for item in provider_input) + + +async def test_generic_mcp_does_not_translate_toolbox_consent() -> None: + mcp = ToolboxConsentServer() + async with ( + httpx.AsyncClient(transport=httpx.MockTransport(mcp.handle)) as http_client, + MCPStreamableHTTPTool(name="generic", url="https://toolbox.invalid/mcp", http_client=http_client) as generic, + ): + with pytest.raises(ToolExecutionException): + await generic.call_tool("learn___protected") + + +async def test_toolbox_oauth_consent_background_run_reconciles_in_foreground() -> None: + mcp = ToolboxConsentServer() + model = ConsentModelServer() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source="inference_service", + background_source="inference_service", + response_store=InMemoryResponseProvider(), + ) + response, _ = await post_consent_response(host, {"input": "Run the tools", "background": True}) + assert response["status"] == "incomplete", response + assert any(item["type"] == "oauth_consent_request" for item in response["output"]) + assert len(model.requests) == 2 + assert model.requests[0]["background"] is True + assert model.requests[1]["background"] is False + assert model.requests[1]["tool_choice"] == "none" + assert [item["type"] for item in model.requests[1]["input"]] == ["function_call_output"] * 3 + + +@pytest.mark.parametrize("stream", [False, True]) +@pytest.mark.parametrize("history_source", ["host_managed", "inference_service"]) +async def test_toolbox_oauth_consent_collects_requests_and_preserves_sibling_results( + stream: bool, history_source: Literal["host_managed", "inference_service"] +) -> None: + mcp = ToolboxConsentServer() + second_link = "https://calendar.example.com/authorize" + mcp.error["message"] = "tools/call failed " + json.dumps({ + "errors": [ + {"name": "learn", "type": "mcp", "error": {"code": "CONSENT_REQUIRED", "message": CONSENT_LINK}}, + {"name": "calendar", "type": "mcp", "error": {"code": "CONSENT_REQUIRED", "message": second_link}}, + ] + }) + model = ConsentModelServer() + async with toolbox_connection(mcp) as toolbox, consent_model_client(model) as chat_client: + host = ResponsesHostServer( + Agent(client=chat_client, tools=toolbox), + history_source=history_source, + response_store=InMemoryResponseProvider(), + ) + response, _ = await post_consent_response(host, {"input": "Run the tools", "stream": stream}) + assert response["status"] == "incomplete", response + assert [item["consent_link"] for item in response["output"] if item["type"] == "oauth_consent_request"] == [ + CONSENT_LINK, + second_link, + ] + results = [item for item in response["output"] if item["type"] == "function_call_output"] + assert len(results) == 3 + assert results[0]["output"] == results[2]["output"] == "search-result" + assert "OAuth consent is required" in results[1]["output"] + assert [call["name"] for call in mcp.calls] == ["learn___completed", "learn___protected", "learn___later"] diff --git a/python/samples/01-get-started/07_hosting.py b/python/samples/01-get-started/07_hosting.py index 6867989f1db..66243718a43 100644 --- a/python/samples/01-get-started/07_hosting.py +++ b/python/samples/01-get-started/07_hosting.py @@ -30,7 +30,7 @@ def main() -> None: instructions="You are a friendly assistant. Keep your answers brief.", default_options={"store": False}, ) - ResponsesHostServer(agent=agent, history_source="agent_server").run() + ResponsesHostServer(agent=agent, history_source="host_managed").run() if __name__ == "__main__": diff --git a/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/README.md b/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/README.md index a3f18afae64..1d6e3174415 100644 --- a/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/README.md +++ b/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/README.md @@ -107,7 +107,7 @@ conversation or MAF session ID. File memory is **sandbox-specific**, unlike the intentional user-wide sharing in the [Foundry Memory sample](../../../../04-hosting/foundry-hosted-agents/responses/foundry_memory/). -`ResponsesHostServer(agent=create_agent, history_source="agent_server")` builds +`ResponsesHostServer(agent=create_agent, history_source="host_managed")` builds new clients, Toolbox/skills providers, CodeAct and harness providers for each request. The sample injects its own context-managed client into `build_claw_agent`, leaving ownership of supplied/shared clients elsewhere diff --git a/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/hosted.py b/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/hosted.py index 139a961d377..110e3f2fa0e 100644 --- a/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/hosted.py +++ b/python/samples/02-agents/harness/build_your_own_claw/claw_step04_production_ready/hosted.py @@ -64,7 +64,7 @@ agent_middleware, ) from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import FoundryRequestScope, ResponsesHostServer +from agent_framework_foundry_hosting import FoundryRequestScope, ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv @@ -102,7 +102,7 @@ def _log_environment() -> None: ) -async def create_agent() -> Agent: +async def create_agent(*, request: ResponsesRequest) -> Agent: """Build request-owned tools and scope file memory by trusted user and sandbox.""" config, context = AgentConfig.from_env(), get_request_context() scope = FoundryRequestScope.from_context(config, context, local_session_id="local-development") @@ -232,7 +232,7 @@ async def main() -> None: _configure_logging() load_dotenv() _log_environment() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/README.md b/python/samples/04-hosting/foundry-hosted-agents/README.md index a7d0bb8446d..db81091f2dd 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/README.md @@ -28,7 +28,7 @@ This directory contains samples that demonstrate how to use hosted [Agent Framew | 16 | [Using deployed agent](responses/using_deployed_agent.py) | Invoke an agent already deployed to Foundry using either a service-created or user-created hosted session, then delete the session after use. | The integration examples use **request-owned factories** and the current -`history_source="agent_server"` API. Use the current workspace, or a hosting +`history_source="host_managed"` API. Use the current workspace, or a hosting release containing that API; before the coordinated beta is published, older PyPI wheels are not evidence that the examples include these behaviors. Toolbox/skills connections inherit the current request's call ID and are closed diff --git a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md index 37283c6edce..a5bf15b3ba9 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/README.md @@ -2,7 +2,7 @@ [`main.py`](main.py) hosts an Agent Framework agent using the Foundry Invocations protocol. The sample accepts application JSON with a `prompt` -field rather than the host's default `message` field. Its `parse_request` +field rather than the host's default `message` field. Its `parse_input` callback returns `InvocationRun(messages, options, stream)`, and `prepare_options` allows only `temperature` and `max_tokens` from the caller. The host also rejects platform IDs, `store`, and private continuation options @@ -85,7 +85,7 @@ when deploying this example to Foundry. [`workflow.py`](workflow.py) is an alternative entrypoint, not a wrapper around the agent above. It hosts a native `Workflow` with -`InvocationsHostServer(workflow=build_workflow, parse_request=parse_request)`; +`InvocationsWorkflowHostServer(workflow=build_workflow, parse_input=parse_input)`; there is no `workflow.as_agent()`. Its deterministic ticket/review executors need no model deployment or Azure credentials locally. @@ -156,6 +156,7 @@ to the agent entrypoint. ### Checkpoints, fresh factories and pending replies +`InvocationsWorkflowHostServer` requires a request-aware factory, not a built workflow. `build_workflow(request)` builds fresh executors on every request. The graph name `ticket-workflow` and executor ID `ticket_start` remain stable. Typed `TicketState` is saved in workflow state through `ctx.set_state`, and restored diff --git a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/main.py b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/main.py index b81f703f1b0..850788538ef 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/main.py @@ -18,7 +18,7 @@ # 1. Map the application's JSON payload to typed MAF inputs. -async def parse_request(request: Request) -> InvocationRun: +async def parse_input(request: Request) -> InvocationRun: """Map application-specific JSON to a validated agent turn.""" payload: Any = await request.json() if not isinstance(payload, dict) or not isinstance(payload.get("prompt"), str): @@ -55,7 +55,7 @@ def main() -> None: server = InvocationsHostServer( agent, - parse_request=parse_request, + parse_input=parse_input, prepare_options=prepare_options, unsupported_options="error", ) diff --git a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/workflow.py b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/workflow.py index 87c0e7a0c80..381bb3e1dce 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/workflow.py +++ b/python/samples/04-hosting/foundry-hosted-agents/invocations/basic/workflow.py @@ -14,7 +14,7 @@ from typing import Any, Literal from agent_framework import Executor, Workflow, WorkflowBuilder, WorkflowContext, handler, response_handler -from agent_framework_foundry_hosting import CheckpointStoreProvider, InvocationsHostServer, WorkflowTurn +from agent_framework_foundry_hosting import CheckpointStoreProvider, InvocationsWorkflowHostServer, WorkflowTurn from starlette.requests import Request """Host typed tickets and durable review replies as a native Invocations workflow. @@ -103,7 +103,7 @@ def unique_json_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]: return values -async def parse_request(request: Request) -> WorkflowTurn[Ticket]: +async def parse_input(request: Request) -> WorkflowTurn[Ticket]: payload: Any = json.loads(await request.body(), object_pairs_hook=unique_json_object) if not isinstance(payload, dict): raise ValueError("The request must be a JSON object.") @@ -150,9 +150,9 @@ def main() -> None: for value in (Ticket, TicketState, TicketReview, TicketDecision, TicketResult) ] ) - InvocationsHostServer( + InvocationsWorkflowHostServer( workflow=build_workflow, - parse_request=parse_request, + parse_input=parse_input, checkpoint_store_provider=checkpoints, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md index 85c465fc6e1..0209037e016 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/README.md @@ -18,7 +18,7 @@ See [main.py](main.py) for the full implementation. each request. The sample's context-managed client explicitly enters/closes the provider and closes its own SDK transports and credential, including failed entry and cancellation; `Agent` does not automatically enter every provider. -`history_source="agent_server"` owns conversation history and disables inner +`history_source="host_managed"` owns conversation history and disables inner model storage. Only first-party Foundry model calls receive the platform call ID; it is not forwarded to Search as an authorization substitute. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/main.py index b029727e9ec..1f561660d01 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/azure_search_rag/main.py @@ -25,13 +25,13 @@ from agent_framework import Agent from agent_framework.azure import AzureAISearchContextProvider from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Allocate a new Search provider and close its transports after this request.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -88,7 +88,7 @@ async def __aexit__( async def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md index d9b46fcd7f0..479856271ce 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/README.md @@ -5,18 +5,18 @@ who supplies the agent's model history: | Entry point | `history_source` | Model history | | --- | --- | --- | -| [main.py](main.py) | `"agent_server"` | The outer Responses transcript; the inner client runs with `store=False`. | -| [service_history.py](service_history.py) | `"service"` | Only new input goes to the model; its private `service_session_id` persists for later turns. | -| [agent_history.py](agent_history.py) | `"agent"` | This agent opts into `InMemoryHistoryProvider` with `default_options={"store": False}`; another `"agent"` configuration with `store=True` can use service history instead. | -| [options.py](options.py) | `"agent_server"` | The hook removes the caller's token limit so the agent default is used. | -| [provider_background.py](provider_background.py) | `"service"` | `background_source="provider"` opts into provider background with a private recovery token. | +| [main.py](main.py) | `"host_managed"` | The outer Responses transcript; the inner client runs with `store=False`. | +| [service_history.py](service_history.py) | `"inference_service"` | Only new input goes to the model; its private `service_session_id` persists for later turns. | +| [agent_history.py](agent_history.py) | `"agent_managed"` | This agent opts into `InMemoryHistoryProvider` with `default_options={"store": False}`; another `"agent_managed"` configuration with `store=True` can use service history instead. | +| [options.py](options.py) | `"host_managed"` | The hook removes the caller's token limit so the agent default is used. | +| [provider_background.py](provider_background.py) | `"inference_service"` | `background_source="inference_service"` opts into provider background with a private recovery token. | For the same two stored requests—first **"My name is Ada"**, then **"What is my name?"** with the first response's `previous_response_id`—the modes differ at the *model* boundary. `main.py` replays both the first user message and the first assistant output before the follow-up. `service_history.py` sends only the follow-up and privately resumes the service thread returned by the first call. `agent_history.py` sends the follow-up plus earlier messages loaded from `InMemoryHistoryProvider` -inside the persisted MAF session. With `history_source="agent"`, the agent's storage default determines +inside the persisted MAF session. With `history_source="agent_managed"`, the agent's storage default determines whether it uses that provider or downstream service storage; the host does not force either one on stored requests. In none of these cases is the caller's `response.id` the downstream service ID. @@ -70,7 +70,7 @@ Send another request with the same `conversation` to continue. When deployed, ke `AgentSession.service_session_id`. A `conversation` binds that sandbox; with a bare `previous_response_id`, also forward the earlier response's `agent_session_id`. Older unscoped hosted MAF state is not migrated; start a new conversation after upgrading (see the [package state guide](../../../../../packages/foundry_hosting/README.md#state-store)). -For `"service"` and `"agent"` history, a stored named turn claims the conversation before inner dispatch: concurrent +For `"inference_service"` and `"agent_managed"` history, a stored named turn claims the conversation before inner dispatch: concurrent turns fail rather than mutating the same service thread. An invalid input does not consume a `previous_response_id` parent. If a named turn fails or is cancelled after dispatch, start a new conversation; the existing thread may have changed even if its outer response did not complete. @@ -85,9 +85,9 @@ removing one exposes the agent's own unchanged `default_options`. In [options.py the model receives the agent's `max_tokens=256` default instead. Platform IDs, storage flags, and private continuation tokens are never caller model options; the hook cannot add them back. -`history_source="agent_server"` and `history_source="agent"` retain their existing meaning without a -deprecation warning. For **stored** requests, `"agent"` passes only new input to the agent, whose +`history_source="host_managed"` and `history_source="agent_managed"` retain their existing meaning without a +deprecation warning. For **stored** requests, `"agent_managed"` passes only new input to the agent, whose developer-owned defaults may choose either a HistoryProvider **or downstream service storage**. -`history_source="service"` explicitly requires service-managed history and overrides the agent's -`store` default on stored requests. `store=` as a constructor parameter remains a deprecated alias -for `response_store=` (the outer storage *backend*, not the caller's `store` flag). +`history_source="inference_service"` explicitly requires service-managed history and overrides the agent's +`store` default on stored requests. The constructor accepts `response_store=` for the outer storage +*backend*, not `store=`. The removed alias does not affect the caller's per-request `store` flag. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/agent_history.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/agent_history.py index 3d4712f01df..0e001924395 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/agent_history.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/agent_history.py @@ -24,7 +24,7 @@ def main() -> None: context_providers=[InMemoryHistoryProvider()], default_options={"store": False}, ) - ResponsesHostServer(agent=agent, history_source="agent").run() + ResponsesHostServer(agent=agent, history_source="agent_managed").run() if __name__ == "__main__": diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/main.py index 2a5158046ff..6f9221850e4 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/main.py @@ -25,7 +25,7 @@ def main() -> None: instructions="You are a friendly assistant. Keep your answers brief.", ) - server = ResponsesHostServer(agent=agent, history_source="agent_server") + server = ResponsesHostServer(agent=agent, history_source="host_managed") server.run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py index f78719b9bd4..681e27d1849 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/options.py @@ -8,14 +8,14 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient from agent_framework.openai import OpenAIChatOptions -from agent_framework_foundry_hosting import HostedResponseRequest, ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.identity import DefaultAzureCredential from dotenv import load_dotenv load_dotenv() -def prepare_options(_request: HostedResponseRequest, options: dict[str, Any]) -> dict[str, Any]: +def prepare_options(_request: ResponsesRequest, options: dict[str, Any]) -> dict[str, Any]: """Use the agent's output-token default instead of the caller's override.""" options.pop("max_tokens", None) return options @@ -33,7 +33,7 @@ def main() -> None: ) ResponsesHostServer( agent=agent, - history_source="agent_server", + history_source="host_managed", prepare_options=prepare_options, unsupported_options="warn", ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py index dc8b8d71b9f..525dd45a00a 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/provider_background.py @@ -6,15 +6,14 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import ResponsesHostServer -from azure.ai.agentserver.responses import ResponsesServerOptions +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.identity import DefaultAzureCredential from dotenv import load_dotenv load_dotenv() -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: return Agent( client=FoundryChatClient( project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], @@ -29,9 +28,9 @@ def create_agent() -> Agent: def main() -> None: ResponsesHostServer( agent=create_agent, - history_source="service", - background_source="provider", - options=ResponsesServerOptions(resilient_background=True), + history_source="inference_service", + background_source="inference_service", + resilient_background=True, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/service_history.py b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/service_history.py index 658c5752c62..6e0483852e1 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/basic/service_history.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/basic/service_history.py @@ -23,7 +23,7 @@ def main() -> None: instructions="Be concise.", default_options={"store": True}, ) - ResponsesHostServer(agent=agent, history_source="service").run() + ResponsesHostServer(agent=agent, history_source="inference_service").run() if __name__ == "__main__": diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md index 340bf9b4cf3..1a812e7f6a4 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/README.md @@ -1,8 +1,9 @@ # Custom session storage (Responses protocol) This sample implements custom **MAF agent-session persistence**, including user -and sandbox isolation and optimistic concurrency. The outer Responses store and -function-approval store remain host-managed. This is not a workflow example. +and sandbox isolation and optimistic concurrency. Native approval state is part +of the same `AgentSession` snapshot. The outer Responses store remains +host-managed; there is no separate function-approval store. This is not a workflow example. ## Identity and storage keys @@ -87,7 +88,7 @@ curl -X POST http://localhost:8088/responses \ -d '{"input":"Hi"}' ``` -The host uses `history_source="agent_server"` and a request-owned agent/client +The host uses `history_source="host_managed"` and a request-owned agent/client factory. It reconstructs history from the outer transcript rather than retaining it on Python instances or enabling inner model storage. Outer `store=false` does not access/write this custom session store. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py index 0b0df183516..0435acc9ef6 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/custom_storage/main.py @@ -36,7 +36,7 @@ from agent_framework import Agent, AgentSession, SessionStore from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import FoundryRequestScope, ResponsesHostServer, StoreProvider +from agent_framework_foundry_hosting import FoundryRequestScope, ResponsesHostServer, ResponsesRequest, StoreProvider from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext, get_request_context from azure.core import MatchConditions from azure.cosmos.aio import ContainerProxy, CosmosClient @@ -220,7 +220,7 @@ async def close(self) -> None: cleanup.push_async_callback(client.close) -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Create and clean up the request's own Foundry transports and credential.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -257,7 +257,7 @@ async def main() -> None: load_dotenv() provider = CustomSessionStoreProvider() server = ResponsesHostServer( - agent=create_agent, history_source="agent_server", agent_session_store_provider=provider + agent=create_agent, history_source="host_managed", agent_session_store_provider=provider ) server.shutdown_handler(provider.close) try: diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md index d447407a786..a42f6126995 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/README.md @@ -3,7 +3,7 @@ [`workflow.yaml`](workflow.yaml) defines a multi-turn customer-support flow using `InvokeAzureAgent`, `ConditionGroup`, `SendActivity`, and Power Fx expressions. [`main.py`](main.py) hosts the built declarative `Workflow` -directly through `ResponsesHostServer(workflow=..., parse_response=...)`. +directly through `ResponsesWorkflowHostServer(workflow=..., parse_input=...)`. The parser uses the explicit `response_input_messages()` migration helper for the declarative workflow's `list[Message]` start contract. The host restores the diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py index 73469924197..7a43d92bcb0 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/declarative_customer_support/main.py @@ -24,8 +24,8 @@ from agent_framework.foundry import FoundryChatClient from agent_framework_declarative import WorkflowFactory from agent_framework_foundry_hosting import ( - HostedResponseRequest, - ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, WorkflowTurn, response_input_messages, ) @@ -80,7 +80,7 @@ async def close(self) -> None: self.credential.close() -def build_workflow(request: HostedResponseRequest) -> Workflow: +def build_workflow(request: ResponsesRequest) -> Workflow: """Load a fresh declarative graph with fresh request-owned agents and stable YAML IDs.""" triage = SupportAgent( "TriageAgent", @@ -105,14 +105,14 @@ def build_workflow(request: HostedResponseRequest) -> Workflow: return factory.create_workflow_from_yaml_path(str(Path(__file__).parent / "workflow.yaml")) -async def parse_response(request: HostedResponseRequest) -> WorkflowTurn[list[Message]]: +async def parse_input(request: ResponsesRequest) -> WorkflowTurn[list[Message]]: """Use the explicit current-turn message migration helper; checkpoints retain prior workflow state.""" return WorkflowTurn(input=await response_input_messages(request)) def main() -> None: load_dotenv() - ResponsesHostServer(workflow=build_workflow, parse_response=parse_response).run() + ResponsesWorkflowHostServer(workflow=build_workflow, parse_input=parse_input).run() if __name__ == "__main__": diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md index 58f75b89784..9efc115b127 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/files/README.md @@ -24,7 +24,7 @@ Set `FOUNDRY_PROJECT_ENDPOINT` and `FOUNDRY_MODEL`, plus either see the [Toolbox sample](../foundry_toolbox/). Authenticate local runs with `az login`. Deployed runs use the sandbox's managed identity. -`ResponsesHostServer(agent=create_agent, history_source="agent_server")` creates +`ResponsesHostServer(agent=create_agent, history_source="host_managed")` creates fresh clients and a Toolbox MCP connection for each request and closes their transports afterward. This request-owned lifecycle gives the sample a clear cleanup boundary; `FoundryToolbox` also rebinds its MCP session to current diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/files/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/files/main.py index 942053882cf..9abe439cfb8 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/files/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/files/main.py @@ -23,7 +23,7 @@ from agent_framework import Agent, tool from agent_framework.foundry import FoundryChatClient, FoundryToolbox -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv @@ -45,7 +45,7 @@ def read_file(filename: str) -> str: return read_uploaded_file(filename) -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Create the client and Toolbox as request-owned resources.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -86,7 +86,7 @@ async def __aexit__( async def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md index bdafbc199cd..315e3d92308 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/README.md @@ -16,7 +16,7 @@ The agent uses `FoundryChatClient` from the Agent Framework to create a Response 2. **Searches for contextual memories** matching the current user message and injects them into the model context. 3. **Updates the store** with new facts inferred from the conversation. -The zero-argument `create_agent` factory builds a new agent, credential, project +The request-aware `create_agent` factory builds a new agent, credential, project client and Memory provider **inside each request**. Chat and Memory share that request's project client, not a process-wide client. Its headers capture the current platform call ID for both Memory and model calls; raw user IDs are not @@ -38,7 +38,7 @@ name is deliberately not part of the hash. Configure separate Memory Stores when agents must not share a user's long-term memories; do not assume sandbox or agent names provide that boundary. -`history_source="agent_server"` supplies conversation history from the outer +`history_source="host_managed"` supplies conversation history from the outer Responses service and disables inner model storage. This is separate from application-owned long-term Memory: outer `store=false` disables host-managed state, **not** the Memory provider's deliberate read/write side effects. Apply diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/main.py index 381f1a00d39..784f44cb689 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_memory/main.py @@ -26,7 +26,7 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient, FoundryMemoryProvider -from agent_framework_foundry_hosting import FoundryRequestScope, ResponsesHostServer +from agent_framework_foundry_hosting import FoundryRequestScope, ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, FoundryAgentRequestContext, get_request_context from azure.ai.projects.aio import AIProjectClient from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential @@ -50,7 +50,7 @@ def memory_scope(config: AgentConfig, context: FoundryAgentRequestContext) -> st return hashlib.sha256(identity.encode("utf-8")).hexdigest() -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Bind a fresh provider and first-party project client to this request.""" config, context = AgentConfig.from_env(), get_request_context() scope = memory_scope(config, context) @@ -92,7 +92,7 @@ async def __aexit__( async def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md index 78393322a1f..1d1962379c8 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/README.md @@ -108,7 +108,7 @@ not retain an earlier caller's `x-agent-foundry-call-id`. The sample client closes only its own model/project transports and credential, including when MCP entry fails or the request is cancelled. Local authentication uses `AzureCliCredential`; deployed authentication uses managed identity. -`history_source="agent_server"` reconstructs the outer Responses transcript and +`history_source="host_managed"` reconstructs the outer Responses transcript and disables inner model storage. Caller `store=false` prevents host-managed state, not external tool side effects. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/main.py index 905d8a3d356..3502974b752 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox/main.py @@ -23,13 +23,13 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient, FoundryToolbox -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Create request-owned SDK resources and close them with the agent.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -67,7 +67,7 @@ async def __aexit__( async def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md index 1ea8ba36653..509ad01d419 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/README.md @@ -43,7 +43,7 @@ arbitrary skill scripts or external actions. The factory's client context closes its owned SDK transports and credential after completion, failed tool entry or cancellation. Local runs use `AzureCliCredential`; hosted runs use managed identity. -`history_source="agent_server"` owns conversation history and disables inner +`history_source="host_managed"` owns conversation history and disables inner model storage. Scope the configured Toolbox/skill resources and RBAC to the intended project; a fresh provider is not an external authorization boundary. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/main.py index 208e231fc92..9d90cde113b 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/foundry_toolbox_mcp_skills/main.py @@ -23,13 +23,13 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient, FoundryToolbox -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Keep skill caches, credentials and the MCP lifecycle request-owned.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -71,7 +71,7 @@ async def __aexit__( async def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md index 479f548be6c..bd619918241 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/README.md @@ -13,7 +13,7 @@ See [main.py](main.py) for the full implementation. `agent=create_agent` creates fresh client and hosted-MCP tool configuration for each request and closes the request's own SDK transports and credential. Local model authentication uses `AzureCliCredential`; deployed calls use managed -identity. `history_source="agent_server"` supplies the outer transcript and +identity. `history_source="host_managed"` supplies the outer transcript and disables inner model storage. **Configure `GITHUB_PAT` separately.** A missing/empty PAT fails explicitly; diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/main.py index 6cc713aa7bf..236178d0808 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/mcp/main.py @@ -22,13 +22,13 @@ from agent_framework import Agent from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Build fresh tool configuration without accepting caller-supplied credentials.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -75,7 +75,7 @@ async def __aexit__( def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") server.run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md index 1d5751c7d6a..7bf5d5fb7e8 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/README.md @@ -21,7 +21,7 @@ events) and non-streaming (JSON) response modes. See [main.py](main.py) for the full implementation. -`ResponsesHostServer(agent=create_agent, history_source="agent_server")` +`ResponsesHostServer(agent=create_agent, history_source="host_managed")` constructs a fresh CodeAct provider, client and credential for each request. No interpreter/provider instance, tool connection or model client is reused from an earlier caller. The factory's client context closes its owned SDK diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/main.py index 67cb53086ea..636aedbe615 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/monty_codeact/main.py @@ -25,7 +25,7 @@ from agent_framework import Agent, tool from agent_framework.foundry import FoundryChatClient from agent_framework.monty import MontyCodeActProvider -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv @@ -70,7 +70,7 @@ def fetch_data( return data.get(table, []) -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Create a new interpreter provider and client for the current request.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -122,7 +122,7 @@ async def __aexit__( def main() -> None: """Host a Monty CodeAct agent over the Responses protocol.""" load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") server.run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md index 39d0893edf9..4801a2ac3fd 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/observability/README.md @@ -28,7 +28,7 @@ OpenTelemetry context propagation links spans to the incoming request rather than a preceding caller. The context-managed sample client closes its own SDK transports and credential after the request. Local authentication uses `AzureCliCredential`; hosted authentication uses managed identity. -`history_source="agent_server"` owns the outer transcript and disables inner +`history_source="host_managed"` owns the outer transcript and disables inner model storage. Foundry Hosted Agent has built-in observability, so the hosted runtime manages diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/observability/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/observability/main.py index 03e035382c2..d61cb73e1a9 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/observability/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/observability/main.py @@ -25,7 +25,7 @@ from agent_framework import Agent, tool from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv @@ -48,7 +48,7 @@ def get_weather( return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C." -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Let incoming telemetry context parent each freshly constructed agent.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -86,7 +86,7 @@ async def __aexit__( async def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") await server.run_async() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md index a373835f484..c9d44c19218 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/README.md @@ -1,6 +1,7 @@ # Resilient native Responses workflow -This sample runs a typed countdown as a native workflow and opts into the +This sample uses `ResponsesWorkflowHostServer` with a required request-aware factory +and parser, runs a typed countdown as a native workflow, and opts into the AgentServer resilient task subsystem. Each acknowledged MAF checkpoint is paired with the exact caller-visible Responses output and usage produced from it. Recovery therefore resumes the same outer `response.id` from that pair rather diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py index 6ba66ef1eb1..bf8ad6125cf 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/resilient_long_running_workflow/main.py @@ -10,12 +10,11 @@ from agent_framework import Executor, Workflow, WorkflowBuilder, WorkflowContext, handler from agent_framework_foundry_hosting import ( CheckpointStoreProvider, - HostedResponseRequest, - ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, WorkflowTurn, ) from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled -from azure.ai.agentserver.responses import ResponsesServerOptions from pydantic import BaseModel, Field @@ -48,7 +47,7 @@ async def countdown(self, target: int, ctx: WorkflowContext[int, str]) -> None: await ctx.send_message(target - 1, target_id=self.id) -def build_workflow(request: HostedResponseRequest) -> Workflow: +def build_workflow(request: ResponsesRequest) -> Workflow: """Return freshly built executors with stable IDs for every invocation and recovery.""" start, countdown = StartExecutor(), CountdownExecutor() return ( @@ -59,20 +58,20 @@ def build_workflow(request: HostedResponseRequest) -> Workflow: ) -async def parse_response(request: HostedResponseRequest) -> WorkflowTurn[CountdownRequest]: +async def parse_input(request: ResponsesRequest) -> WorkflowTurn[CountdownRequest]: return WorkflowTurn(input=CountdownRequest.model_validate_json(await request.get_input_text() or "")) def main() -> None: # The application's explicit SDK task opt-in is separate from output checkpointing. set_resilient_tasks_enabled(True) - ResponsesHostServer( + ResponsesWorkflowHostServer( workflow=build_workflow, - parse_response=parse_response, + parse_input=parse_input, checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{CountdownRequest.__module__}:{CountdownRequest.__qualname__}"], ), - options=ResponsesServerOptions(resilient_background=True), + resilient_background=True, ).run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md index 18b2388fb71..5d4214b67d5 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/README.md @@ -1,16 +1,16 @@ # Long-running Responses agent (steering temporarily unavailable) -**Steering is currently gated.** `ResponsesHostServer` rejects -`ResponsesServerOptions(steerable_conversations=True)` during construction rather than starting a -TaskManager that can retain unbounded futures after rejected steering requests. The upstream fix is +**Steering is not exposed.** `ResponsesHostServer` has no `options` or `steerable_conversations` +parameter, and framework-generated SDK options keep steering disabled. The SDK TaskManager can retain +unbounded futures after rejected steering requests. The upstream fix is [Azure/azure-sdk-for-python#49233](https://github.com/Azure/azure-sdk-for-python/pull/49233); -steering will be re-enabled only after a patched official `azure-ai-agentserver-core` wheel is released, -the package's minimum dependency and lockfile are updated, and a concurrent overflow test verifies -that rejected turns leave no pending futures. Do not enable steering using a private SDK patch or a +any future steering API needs a patched official `azure-ai-agentserver-core` wheel, updated minimum +dependencies and lockfile, and a concurrent overflow test proving that rejected turns leave no pending futures. +Do not enable steering using a private SDK patch or a queue-length precheck. [main.py](main.py) remains **runnable** as a regular non-steerable countdown agent. It uses -`history_source="agent_server"` with a `FoundryChatClient` and returns the caller's **outer** `response.id` for +`history_source="host_managed"` with a `FoundryChatClient` and returns the caller's **outer** `response.id` for background polling. Running a second turn concurrently does not steer the first. Start it using the [parent hosting guide](../../README.md), then send one stored background request. The deployment manifests keep their existing agent name for identity compatibility, not because steering is enabled: diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py index 42c7db60aa2..b6e3f20d040 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/main.py @@ -4,8 +4,8 @@ The agent is asked to count down from a target number, pacing its own output with a short remark before each number so a response takes a while to fully generate. This sample currently runs -without steering; the hosting layer rejects ``steerable_conversations=True`` until a patched -AgentServer SDK is released and verified. +without steering; ``steerable_conversations`` is not a host parameter and framework-generated +SDK options keep steering disabled. Environment variables: FOUNDRY_PROJECT_ENDPOINT: Microsoft Foundry project endpoint. @@ -42,7 +42,7 @@ def main() -> None: server = ResponsesHostServer( agent=agent, - history_source="agent_server", + history_source="host_managed", log_level="DEBUG", ) server.run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py index cfe7b1d47e0..964d3b9ec2c 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/steerable_long_running_agent/verify_steering.py @@ -1,28 +1,30 @@ # Copyright (c) Microsoft. All rights reserved. -"""Verify that steering fails before starting a host while the AgentServer SDK is affected. +"""Verify that the unsupported steering setting cannot start an agent or host. -This script does not require Azure credentials or make model requests. The full end-to-end -steering verifier must be restored when a patched core wheel and overflow regression are in place. +This script does not require Azure credentials or make model requests. A future steering verifier +requires a supported steering API, a patched core wheel, and an overflow regression. Expected output: Steering is temporarily unavailable; no agent or host was started. """ +from typing import Any + from agent_framework import SupportsAgentRun -from agent_framework_foundry_hosting import ResponsesHostServer -from azure.ai.agentserver.responses import ResponsesServerOptions +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest -def _agent_factory() -> SupportsAgentRun: +def _agent_factory(*, request: ResponsesRequest) -> SupportsAgentRun: raise AssertionError("A guarded steering host must not create an agent.") def main() -> None: """Confirm that the constructor rejects steering before creating an agent or host.""" + removed_settings: dict[str, Any] = {"steerable_conversations": True} try: - ResponsesHostServer(agent=_agent_factory, options=ResponsesServerOptions(steerable_conversations=True)) - except RuntimeError as exc: - if "steerable_conversations=True is temporarily unavailable" not in str(exc): + ResponsesHostServer(agent=_agent_factory, **removed_settings) + except TypeError as exc: + if "steerable_conversations" not in str(exc): raise print("Steering is temporarily unavailable; no agent or host was started.") return diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md index 82da154db63..49a1d4b3da0 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/tools/README.md @@ -24,7 +24,7 @@ When a tool is set to `always_require`, the agent host emits an `mcp_approval_re The agent is hosted using the [Agent Framework](https://github.com/microsoft/agent-framework) with the `ResponsesHostServer`, which provisions a REST API endpoint compatible with the OpenAI Responses protocol. -The host uses `agent=create_agent` and `history_source="agent_server"`. Each +The host uses `agent=create_agent` and `history_source="host_managed"`. Each request receives its own client and credential; their owned transports are closed afterward. History and approval continuation remain in the host's trusted user/sandbox-scoped stores, not on a reused agent instance. A fresh diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/tools/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/tools/main.py index e77f4ace500..45794cbf392 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/tools/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/tools/main.py @@ -25,7 +25,7 @@ from agent_framework import Agent, tool from agent_framework.foundry import FoundryChatClient -from agent_framework_foundry_hosting import ResponsesHostServer +from agent_framework_foundry_hosting import ResponsesHostServer, ResponsesRequest from azure.ai.agentserver.core import AgentConfig, get_request_context from azure.identity.aio import AzureCliCredential, ManagedIdentityCredential from dotenv import load_dotenv @@ -65,7 +65,7 @@ def run_bash(command: str) -> str: return f"Error executing command: {e}" -def create_agent() -> Agent: +def create_agent(*, request: ResponsesRequest) -> Agent: """Create client state per request rather than keeping history on the agent.""" endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"] model = os.environ.get("FOUNDRY_MODEL") or os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] @@ -103,7 +103,7 @@ async def __aexit__( def main() -> None: load_dotenv() - server = ResponsesHostServer(agent=create_agent, history_source="agent_server") + server = ResponsesHostServer(agent=create_agent, history_source="host_managed") server.run() diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/README.md b/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/README.md index 096980b8f58..e1cedf09c2d 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/README.md +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/README.md @@ -1,7 +1,7 @@ # Native Responses workflows This sample hosts built Agent Framework workflows directly with -`ResponsesHostServer(workflow=..., parse_response=...)`; it does not wrap them +`ResponsesWorkflowHostServer(workflow=..., parse_input=...)`; it does not wrap them with `.as_agent()`. - [`main.py`](main.py) parses a typed `SloganRequest`, builds fresh agents, @@ -18,8 +18,8 @@ inside that sandbox. The MAF checkpoint ID is private workflow state and is bound to the exact outer response; none of these IDs is a downstream model `service_session_id`. -Use a request-aware factory for multi-turn workflows or pauses. A built -`Workflow` instance is single-use and is not cloned. Every factory invocation +The workflow host requires a request-aware factory, even for one-shot execution. +A built `Workflow` instance is rejected at construction, not cloned. Every factory invocation must create fresh mutable resources, while workflow names and executor IDs remain stable. `store=False` is one-shot and cannot expose a resumable pause. diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/approval.py b/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/approval.py index 972b72fe1f3..a2856f403a5 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/approval.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/approval.py @@ -18,8 +18,8 @@ ) from agent_framework_foundry_hosting import ( CheckpointStoreProvider, - HostedResponseRequest, - ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, WorkflowTurn, ) from pydantic import BaseModel, Field @@ -55,11 +55,11 @@ async def decide(self, request: Content, response: Content, ctx: WorkflowContext await ctx.yield_output(PublicationResult(summary=ctx.get_state("summary"), approved=response.approved is True)) -def build_workflow(request: HostedResponseRequest) -> Workflow: +def build_workflow(request: ResponsesRequest) -> Workflow: return WorkflowBuilder(name="approved-publication-v1", start_executor=PublicationExecutor()).build() -async def parse_response(request: HostedResponseRequest) -> WorkflowTurn[PublicationRequest]: +async def parse_input(request: ResponsesRequest) -> WorkflowTurn[PublicationRequest]: items = await request.get_input_items() if any(item.get("type") == "mcp_approval_response" for item in items): return WorkflowTurn(responses=await request.get_workflow_responses()) @@ -67,9 +67,9 @@ async def parse_response(request: HostedResponseRequest) -> WorkflowTurn[Publica def main() -> None: - ResponsesHostServer( + ResponsesWorkflowHostServer( workflow=build_workflow, - parse_response=parse_response, + parse_input=parse_input, checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{PublicationRequest.__module__}:{PublicationRequest.__qualname__}"], ), diff --git a/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/main.py b/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/main.py index a45c406ea24..5951e136425 100644 --- a/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/main.py +++ b/python/samples/04-hosting/foundry-hosted-agents/responses/workflows/main.py @@ -21,8 +21,8 @@ from agent_framework.foundry import FoundryChatClient from agent_framework_foundry_hosting import ( CheckpointStoreProvider, - HostedResponseRequest, - ResponsesHostServer, + ResponsesRequest, + ResponsesWorkflowHostServer, WorkflowTurn, ) from azure.identity import DefaultAzureCredential @@ -74,7 +74,7 @@ async def start(self, request: SloganRequest, ctx: WorkflowContext[AgentExecutor ) -def build_workflow(request: HostedResponseRequest) -> Workflow: +def build_workflow(request: ResponsesRequest) -> Workflow: """Build fresh graphs, executors, agents, clients, and credentials with stable graph IDs.""" start = StartExecutor() writer = AgentExecutor( @@ -99,16 +99,16 @@ def build_workflow(request: HostedResponseRequest) -> Workflow: ) -async def parse_response(request: HostedResponseRequest) -> WorkflowTurn[SloganRequest]: +async def parse_input(request: ResponsesRequest) -> WorkflowTurn[SloganRequest]: """Parse only this turn's Responses text; history and checkpoint selection belong to the host.""" return WorkflowTurn(input=SloganRequest.model_validate_json(await request.get_input_text() or "")) def main() -> None: load_dotenv() - ResponsesHostServer( + ResponsesWorkflowHostServer( workflow=build_workflow, - parse_response=parse_response, + parse_input=parse_input, checkpoint_store_provider=CheckpointStoreProvider( allowed_checkpoint_types=[f"{SloganRequest.__module__}:{SloganRequest.__qualname__}"], ), diff --git a/python/scripts/sample_validation/skills/foundry-hosted-agent-validation/SKILL.md b/python/scripts/sample_validation/skills/foundry-hosted-agent-validation/SKILL.md index c21c4ac672d..87ac362b523 100644 --- a/python/scripts/sample_validation/skills/foundry-hosted-agent-validation/SKILL.md +++ b/python/scripts/sample_validation/skills/foundry-hosted-agent-validation/SKILL.md @@ -66,7 +66,8 @@ Gather these: ## Phase 0 — Understand the sample A responses/invocations sample folder typically contains: -`main.py` (entry point + `ResponsesHostServer`/`InvocationsHostServer`), +`main.py` (entry point + agent `ResponsesHostServer`/`InvocationsHostServer`, or native +`ResponsesWorkflowHostServer`/`InvocationsWorkflowHostServer` with a request-aware factory and parser), `agent.manifest.yaml` (used by `azd ai agent init`), `agent.yaml` (the deployed agent definition), `requirements.txt`, `Dockerfile`, `.env.example`. diff --git a/skills/agent-framework-builder-python/references/hosting-and-deployment.md b/skills/agent-framework-builder-python/references/hosting-and-deployment.md index bf6c87344ae..26f0cefe942 100644 --- a/skills/agent-framework-builder-python/references/hosting-and-deployment.md +++ b/skills/agent-framework-builder-python/references/hosting-and-deployment.md @@ -21,7 +21,7 @@ into application code. - Treat translated request options as caller-controlled input and forward only an explicit allowlist. - Choose one history owner; do not combine competing host, service, and agent history mechanisms. - Reuse an agent instance only when all mutable state is session-scoped. Otherwise use a request-owned factory. -- Prefer native workflow hosting. A shared `workflow.as_agent()` instance can leak workflow state across requests. +- Use native workflow hosting. Foundry agent hosts reject `WorkflowAgent` instances and factory results. - Build fresh workflows and request-owned resources while keeping workflow and executor identities stable for checkpoint restoration. A builder reused across requests returns new workflows that share its registered executors, including each `AgentExecutor` and its `AgentSession`; create those executors inside the per-request @@ -45,6 +45,21 @@ choose patterns with predictable matching time for untrusted values and property Read the selected host's public docstrings and maintained sample for constructor options, parsers, history modes, storage providers, and protocol details. +### Foundry agent and native workflow hosts + +Choose an agent host for session-backed agent execution and a native workflow host for checkpoint-owned +execution and pending replies. Do not mix their storage or history policies, or treat a workflow wrapper +as an agent-host substitute. + +Use request-aware factories for identity-dependent resources. Build fresh workflows, executors, agents, +clients, providers, and mutable tools for each workflow request; a new graph around shared executors +does not isolate execution state. Keep graph and executor IDs stable for checkpoint restoration. +Make cleanup ownership explicit and keep persistent state in the appropriate stores. + +Use the maintained [Foundry Hosting package guide](https://github.com/microsoft/agent-framework/blob/main/python/packages/foundry_hosting/README.md) +and [hosting samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents) +for public imports, factory signatures, parser contracts, configuration, and migration details. + ## Deployment readiness Verify the entry point imports, required packages and external files are included, configuration fails clearly, history