Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/specs/004-python-function-calling-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,13 @@ that manually replay messages own the equivalent rule: do not resend an approval
- Parallel calls retain model order in the returned transcript.
- `call_id` remains the provider/service correlation id; a locally actionable `function_call` also carries a stable
Agent Framework occurrence identity in `Content.id`.
- `FunctionInvocationContext.metadata["call_id"]` exposes the provider correlation ID, and
`metadata["function_call_occurrence_id"]` exposes the canonical `Content.id` when function-call content has one.
The automatic loop populates these entries before function middleware and tool execution on both execution
paths, including approval resume. Standalone `FunctionTool.invoke` sets or clears the provider ID from
`tool_call_id` before argument and security validation, including failed invocations. An occurrence ID supplied
with a matching provider call ID is preserved; changing or omitting the provider ID clears the occurrence ID.
Absent identity values leave no corresponding metadata entry. Call payloads are not added to the context.
- Reused `call_id` values are correlated by logical occurrence, not one global value per id.
- Existing `Content.id` values survive aggregation and replay and are never regenerated; legacy deserialization does
not invent one.
Expand Down Expand Up @@ -605,6 +612,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
| String input | Flexible string input follows the same loop behavior. | `test_base_client_with_function_calling_string_input` |
| Multiple sequential rounds | Each round retains one call/result pair. | `test_base_client_with_function_calling_resets` |
| Streaming call | Call chunks, one result update, and final text are emitted in order. | `test_base_client_with_streaming_function_calling` |
| Tool-visible invocation identity metadata | Tools and function middleware receive the provider call ID and canonical occurrence ID on both execution paths, in both response modes, and after approval resume, including reused provider IDs. Standalone invocation updates or clears identity metadata before validation and retains a supplied occurrence only with a matching provider ID. | `test_tool_context_exposes_call_identity_metadata`, `packages/core/tests/core/test_tools.py::test_invoke_exposes_tool_call_id_metadata`, `test_invoke_clears_stale_identity_metadata`, `test_invoke_preserves_matching_occurrence_metadata`, `test_invoke_updates_call_metadata_before_argument_validation`, `test_invoke_updates_call_metadata_before_security_validation` |
| Function-call occurrence identity | Actionable calls gain one stable `Content.id`; a safe local empty-`call_id` fallback uses that id with a migration warning, and streaming aggregation preserves provider-assigned occurrence ids across interleaved fragments. OpenAI Chat Completions scopes fragment correlation to each request and `(choice.index, tool.index)`. | `test_actionable_function_call_gets_stable_occurrence_identity`, `test_actionable_function_call_uses_occurrence_identity_for_empty_call_id`, `test_streaming_empty_call_id_keeps_occurrence_identity_through_approval`, `test_streaming_empty_call_id_delta_reuses_opening_call_identity`, `test_streaming_interleaved_indexed_call_fragments_coalesce_by_occurrence`, `packages/core/tests/core/test_types.py::test_function_call_occurrence_id_roundtrips_without_regeneration`, `packages/openai/tests/openai/test_openai_chat_completion_client.py::test_streaming_tool_call_identity_is_request_local_and_scoped_by_choice_index` |
| Reasoning-bound call | Finalized output retains reasoning, function call, function result, and final text. | `test_streaming_function_calling_response_includes_reasoning_and_tool_results` |
| Calls across response messages | Every actionable call is executed once. | `test_base_client_executes_function_calls_across_multiple_response_messages` |
Expand Down
5 changes: 5 additions & 0 deletions python/packages/core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,11 @@ The vector store API is experimental under the shared `VECTOR_STORES` feature ID
- **`AgentMiddleware`** - Intercepts agent `run()` calls
- **`ChatMiddleware`** - Intercepts chat client `get_response()` calls
- **`FunctionMiddleware`** - Intercepts function/tool invocations
- **Function invocation identity metadata** - `FunctionInvocationContext.metadata["call_id"]` holds the provider
correlation ID, and `metadata["function_call_occurrence_id"]` holds the canonical `Content.id` when present.
The automatic loop supplies both before function middleware and tool execution, including approval resume.
Standalone `FunctionTool.invoke` sets or clears the provider ID from `tool_call_id` before validation and
retains an occurrence ID only when it was supplied with a matching provider call ID.
- **`AgentContext`** / **`ChatContext`** / **`FunctionInvocationContext`** - Context objects passed through middleware. A tool can declare a `FunctionInvocationContext` parameter to receive it; `context.tools` is the live, mutable tools list for the run, and `context.add_tools(...)` / `context.remove_tools(...)` enable progressive tool exposure (changes apply on the next function-calling iteration). Chat middleware that constructs provider-local replacement messages must call `context.record_message_replacement(...)` for every replacement before downstream compaction so complete summaries can reconcile to caller-owned messages without persisting the replacements themselves.
- **`MessageInjectionMiddleware`** - Session-scoped chat middleware that lets tools or other code enqueue messages for the next model call in the current `AgentSession`; it drains queued messages into the next call and loops only when no function calls need to be handled by the function invocation layer.

Expand Down
8 changes: 7 additions & 1 deletion python/packages/core/agent_framework/_middleware.py
Original file line number Diff line number Diff line change
Expand Up @@ -449,7 +449,13 @@ class FunctionInvocationContext:
Middleware that short-circuits without calling ``call_next()`` skips
final validation and function execution.
session: The agent session for this invocation, if any.
metadata: Metadata dictionary for sharing data between function middleware.
metadata: Metadata dictionary shared by function middleware and context-aware tools.
Automatic invocation supplies ``"call_id"`` for provider correlation and
``"function_call_occurrence_id"`` for the canonical ``Content.id``, when present,
before middleware and tool execution, including approval resume. Standalone
``FunctionTool.invoke`` calls set or clear ``"call_id"`` from ``tool_call_id``
before validation. A supplied occurrence ID is retained only when the context's
provider call ID matches the supplied ``tool_call_id``; otherwise it is cleared.
result: Function execution result. This attribute carries no guaranteed type.
The pipeline assigns :meth:`FunctionTool.invoke`'s output — a
``list[Content]``, or the wrapped function's raw return value when the
Expand Down
39 changes: 34 additions & 5 deletions python/packages/core/agent_framework/_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,23 @@
logger = logging.getLogger("agent_framework")


def _update_function_invocation_metadata(
context: FunctionInvocationContext,
*,
call_id: str | None,
occurrence_id: str | None = None,
) -> None:
"""Replace invocation identity metadata without retaining missing values."""
if call_id is None:
context.metadata.pop("call_id", None)
else:
context.metadata["call_id"] = call_id
if occurrence_id is None:
context.metadata.pop("function_call_occurrence_id", None)
else:
context.metadata["function_call_occurrence_id"] = occurrence_id


def _generate_function_call_occurrence_id() -> str:
"""Generate an Agent Framework identity for one function-call occurrence."""
return f"af-call-{uuid4().hex}"
Expand Down Expand Up @@ -1138,7 +1155,11 @@ async def invoke(
Keyword Args:
arguments: A mapping or model instance containing the arguments for the function.
context: Explicit function invocation context carrying runtime kwargs.
tool_call_id: Optional tool call identifier used for telemetry and tracing.
tool_call_id: Optional provider call identifier used for telemetry, tracing, and
``context.metadata["call_id"]``. Omitting it clears the context's call identity.
Occurrence metadata supplied with a matching context call ID is preserved;
changing or omitting the call ID clears it. Identity metadata is updated before
argument and security validation, including when invocation fails.
skip_parsing: When ``True``, bypass parsing and return the wrapped function's
raw value instead of a ``list[Content]``. Defaults to ``False``.
kwargs: Direct function argument values. When provided, every keyword
Expand All @@ -1153,6 +1174,13 @@ async def invoke(
Raises:
TypeError: If arguments is not mapping-like or fails schema checks.
"""
if context is not None:
occurrence_id = (
context.metadata.get("function_call_occurrence_id")
if tool_call_id is not None and context.metadata.get("call_id") == tool_call_id
else None
)
_update_function_invocation_metadata(context, call_id=tool_call_id, occurrence_id=occurrence_id)
if self.declaration_only:
raise ToolException(f"Function '{self.name}' is declaration only and cannot be invoked.")
global OBSERVABILITY_SETTINGS
Expand Down Expand Up @@ -1196,6 +1224,7 @@ async def invoke(
arguments=validated_arguments,
kwargs=runtime_kwargs,
)
_update_function_invocation_metadata(effective_context, call_id=tool_call_id)
if effective_context is not None:
effective_context.function = self
effective_context.arguments = validated_arguments
Expand Down Expand Up @@ -2339,6 +2368,9 @@ async def _auto_invoke_function(
parent_service_session_state_keys=parent_service_session_state_keys,
tools=live_tools,
)
_update_function_invocation_metadata(
direct_context, call_id=function_call_content.call_id, occurrence_id=function_call_content.id
)
direct_context.metadata[_PARENT_TOOL_APPROVAL_SOURCE_IDS_CONTEXT_KEY] = parent_approval_source_ids
if host_payload_budget is not None:
direct_context.metadata[_FUNCTION_RESULT_PAYLOAD_BUDGET_CONTEXT_KEY] = host_payload_budget
Expand Down Expand Up @@ -2394,10 +2426,7 @@ async def _auto_invoke_function(
if call_id is None:
raise KeyError(f'Function "{function_call_content.name}" is missing call_id.')

# Pass both provider correlation and framework occurrence identity to middleware.
middleware_context.metadata["call_id"] = call_id
if function_call_content.id is not None:
middleware_context.metadata["function_call_occurrence_id"] = function_call_content.id
_update_function_invocation_metadata(middleware_context, call_id=call_id, occurrence_id=function_call_content.id)

# Pass through the original approval response so middleware can decide whether
# this replay corresponds to a middleware-specific approval flow.
Expand Down
128 changes: 128 additions & 0 deletions python/packages/core/tests/core/test_function_invocation_logic.py
Original file line number Diff line number Diff line change
Expand Up @@ -971,6 +971,134 @@ def _finalize(stream_updates: Sequence[ChatResponseUpdate]) -> ChatResponse:
chat_client_base._get_streaming_response = _get_streaming_response


@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
@pytest.mark.parametrize("with_middleware", [False, True], ids=["direct", "middleware"])
@pytest.mark.parametrize("requires_approval", [False, True], ids=["immediate", "approved"])
@pytest.mark.parametrize("reused_provider_id", [False, True], ids=["distinct-call-ids", "reused-call-id"])
async def test_tool_context_exposes_call_identity_metadata(
chat_client_base: SupportsChatGetResponse,
streaming: bool,
with_middleware: bool,
requires_approval: bool,
reused_provider_id: bool,
) -> None:
"""Tools and middleware receive provider and occurrence identities across execution paths."""
observed_identities: list[tuple[str, str]] = []
middleware_identities: list[tuple[str, str]] = []
expected_identities = [
("provider-shared" if reused_provider_id else f"provider-{index}", f"occurrence-{index}") for index in (1, 2)
]

class ObserveCall(FunctionMiddleware):
async def process(self, context: FunctionInvocationContext, call_next: Callable[[], Awaitable[None]]) -> None:
middleware_identities.append((context.metadata["call_id"], context.metadata["function_call_occurrence_id"]))
await call_next()

@tool(name="lookup", approval_mode="always_require" if requires_approval else "never_require")
async def lookup(context: FunctionInvocationContext, value: str) -> str:
await asyncio.sleep(0)
observed_identities.append((context.metadata["call_id"], context.metadata["function_call_occurrence_id"]))
return value

calls = [
Content.from_function_call(
call_id="provider-shared" if reused_provider_id else f"provider-{index}",
name="lookup",
arguments='{"value":"same"}',
id=f"occurrence-{index}",
additional_properties={"provider": {"item_id": f"item-{index}"}},
)
for index in (1, 2)
]
responses = [
ChatResponse(messages=Message(role="assistant", contents=calls)),
ChatResponse(messages=Message(role="assistant", contents=["done"])),
]
if streaming:
chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
[
ChatResponseUpdate(
role="assistant",
contents=[
Content.from_function_call(
call_id="provider-shared" if reused_provider_id else f"provider-{index}",
name="lookup",
arguments='{"value":',
id=f"occurrence-{index}",
additional_properties={"provider": {"item_id": f"item-{index}"}},
)
for index in (1, 2)
],
),
ChatResponseUpdate(
role="assistant",
contents=[
Content.from_function_call(
call_id="provider-shared" if reused_provider_id else f"provider-{index}",
name="",
arguments='"same"}',
id=f"occurrence-{index}",
)
for index in (1, 2)
],
),
],
[ChatResponseUpdate(role="assistant", contents=[Content.from_text("done")])],
]
else:
chat_client_base.run_responses = responses # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]

agent = Agent(client=chat_client_base, tools=[lookup], middleware=[ObserveCall()] if with_middleware else [])
session = agent.create_session()

async def run(messages: str | list[Message]) -> list[Message]:
if streaming:
response_stream = agent.run(messages, session=session, stream=True)
updates = [update async for update in response_stream]
response = await response_stream.get_final_response()
assert [
content.call_id
for update in updates
for content in update.contents
if content.type == "function_result"
] == [
content.call_id
for message in response.messages
for content in message.contents
if content.type == "function_result"
]
else:
response = await agent.run(messages, session=session)
return response.messages

messages = await run("look up the same value twice")
if requires_approval:
assert observed_identities == []
assert middleware_identities == []
requests = [
content
for message in messages
for content in message.contents
if content.type == "function_approval_request"
]
assert len(requests) == 2
session = AgentSession.from_dict(json.loads(json.dumps(session.to_dict())))
messages = await run([
Message(
role="user", contents=[request.to_function_approval_response(approved=True) for request in requests]
)
])

assert sorted(observed_identities) == expected_identities
assert middleware_identities == (expected_identities if with_middleware else [])
assert [
(content.call_id, content.id, content.result)
for message in messages
for content in message.contents
if content.type == "function_result"
] == [(provider_call_id, occurrence_id, "same") for provider_call_id, occurrence_id in expected_identities]


async def test_base_client_with_function_calling(chat_client_base: SupportsChatGetResponse):
exec_counter = 0

Expand Down
Loading
Loading