Block provider network escapes in restricted runtimes

[xeophon]

Summary

• enforce each rollout session’s effective runtime network restriction before a native model request reaches the provider
• remove blocked provider-side URLs, hosted tools, MCP, and provider-held state from restricted requests, then add a value-free runtime policy error to the model prompt so it can recover
• cover Responses, Chat Completions, Anthropic Messages, streaming requests, and Anthropic count_tokens
• keep intrinsically networked search models as a typed HTTP 400 configuration failure because their hosted access cannot be removed from an individual request

Security model

Any restricted runtime policy disables provider-side fetches and hosted capabilities. Provider DNS and redirects happen outside the sandbox firewall, so a runtime allowlist cannot safely authorize a provider-side URL.

Mediation removes protocol-defined external capability fields before the upstream call. The model sees only structural field paths in the policy notice; URLs, file IDs, MCP authorization, and other request values are never copied into the notice. The sanitized effective request is also what Verifiers parses and records.

Harness behavior

Recoverable policy violations no longer return an HTTP error to the harness. The provider receives a safe request containing the policy notice, and the harness receives the model’s ordinary response, so all harnesses can continue without dialect-specific error handling. Only a capability intrinsic to the configured model remains a hard error.

Testing

• full local V1 suite passed; PRIME_API_KEY-gated cases skipped
• full local non-V1 suite passed
• Chat, Responses, and Anthropic mediation passed in streaming and non-streaming modes
• Anthropic count_tokens mediation passed
• Ruff format and check passed
• ty check passed
• commit and push hooks passed

The previous live V1 CI failure was an unrelated transient 429 during rubric scoring; 58 other live tests passed.

---

Note

High Risk
Changes the security-critical interception path and request shaping for all dialects; custom Client subclasses must update signatures, and mediation bugs could leak provider fetches or break harnesses under restricted policies.

Overview
Restricted rollouts now strip provider-side network escapes before upstream calls: hosted tools, MCP, remote file/image URLs, web search, and similar fields are removed in Chat, Responses, and Anthropic dialects, with a value-free user notice so the harness still gets a normal model turn. PolicyEvent records affected request paths on each ModelCall.

The interception server applies apply_overrides then mediate_external_capabilities when RolloutSession.network_policy is restricted; clients receive the final body only (model dropped from get_response / relay). NetworkPolicyConfig and network_rule_matches live in configs/runtime; task allowlists intersect with runtime rules instead of widening by union.

Smaller fixes: ACP replayed_transcript for resume vs live turns, Responses terminal failure / streaming context-length handling, Harbor docs on provider capabilities vs runtime egress, and a richer ACP resume fixture.

^{Reviewed by Cursor Bugbot for commit 938a5e9. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Block provider network escapes in restricted runtimes by mediating AI provider requests against rollout network policy

• Adds NetworkPolicyConfig.permits to test egress URLs against allow/block rules, and propagates the resolved policy through RolloutSession into the interception pipeline.
• Implements mediate_external_capabilities on all three dialects (anthropic.py, chat.py, responses.py): removes disallowed tools, content references (images, files, hosted IDs), and provider-side features (web search, MCP servers, audio voices), then injects a policy notice into the earliest user message when removals occur.
• Rollouts targeting models with intrinsic provider-side network capabilities (e.g. built-in search) now fail early with InterceptionError before the request is sent.
• Adds PolicyEvent to ModelCall traces so mediation decisions and affected request paths are recorded alongside each call.
• Refactors Client.get_response/relay signatures: model is removed as a separate parameter and sampling_args is renamed to sampling; callers must supply a fully mediated body. Risk: any custom Client subclass must update its method signatures.

Changes since #2299 opened

• Replaced _HOSTED_TOOL_TYPES frozenset with _HOSTED_TOOL_TYPE compiled regex pattern matching file_search|mcp|code_interpreter|programmatic_tool_calling|image_generation|web_search_preview(?:_\d{4}_\d{2}_\d{2})? to support date-suffixed web_search_preview variants, updated _mediate_tools function to use regex fullmatch for hosted tool detection with string type guard and special handling for tool_search and shell kinds, and modified ResponsesDialect.mediate_external_capabilities method to reject dict tool_choice entries matching hosted-tool or web tool regex patterns and to validate shell or tool_search tool_choice only when matching tool types exist in the tools array [fde99e0]
• Added exception handling in network_rule_matches function to catch ValueError when accessing parsed port and return False for rules with invalid port specifications [fde99e0]
• Renamed module-level constants and helper function from private to public exports [0962376]
• Moved NetworkPolicyConfig class and network_rule_matches function from verifiers.v1.runtimes.base module to verifiers.v1.configs.runtime module [b0cb17b]
• Redesigned capability mediation across all API dialects to unconditionally block provider-resolved remote resources and hosted tools in restricted runtimes [938a5e9]
• Changed network policy composition in NetworkPolicyConfig.with_task_network_policy to retain only exact shared allow rules and always combine blocklists [938a5e9]
• Introduced syntax error in TaskData model by inserting free-form descriptive text between field docstrings outside string literals [938a5e9]
• Enhanced ACP session transcript replay to trim prior assistant messages based on replay status rather than session newness [938a5e9]
• Extended resumed MCP call test to require two codewords in multi-message delta interaction [938a5e9]
• Added typed exception handling for streaming errors in interception server turn creation and enforced rollout model selection for auxiliary endpoints [938a5e9]
• Improved error surfacing in response_from_wire by raising typed exceptions for non-completed response statuses [938a5e9]
• Removed preflight check in Rollout.open that failed rollouts based on intrinsic provider-side model capabilities under restricted policies [938a5e9]
• Refactored URL scheme detection in egress proxy to use string prefix checks instead of URL parsing [938a5e9]
• Fixed ChatDialect.apply_overrides merge order to ensure selected model remains authoritative over sampling payload fields [938a5e9]
• Updated documentation to clarify network policy behavior for allowlist retention, blocklist combination, and provider resource restrictions [938a5e9]

^{Macroscope summarized 4cef434.}

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[macroscopeapp]

Approvability

Verdict: Needs human review

2 blocking correctness issues found. This PR introduces a new security/policy enforcement layer that mediates provider-side capabilities based on network restrictions. The substantial new mediation logic across multiple dialects, combined with unresolved design questions and two High-severity correctness findings, warrants careful human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[hallerite]

important PR to get in. I would prefer if we can return a stable structured error to let the different harnesses optionally recover (we should def not put the onus of crafting a response to let the model know why the request was rejected on the proxy)

otherwise, lgtm!

[mikasenghaas]

hmm i really dont like us being in charge of network restrictions on any external resource (any inference deployment, api, mcp server, etc. etc.). in my opinion this is a battle we really cannot win. plus it adds a huge amount of line noise and unmaintainable ai slop into our codebase. i would much rather maintain a list of known ways to "escape" the sandbox and tune our interception machinery to reliably detect these things

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4cef434. Configure here.}

[mikasenghaas]

can we run some example traces for each external tool with no block and block where we explicitly ask the model to do use the blocked tool and then check that we a) block the tool and b) so its clear what our mediation strategy acc is

also we choose to only care abt external tools through infernece. we deliberately dont try to block e.g. mcp (this is user respnosibility). should we hav a small note on this in the docs?