Skip to content

(core): add function_tools_called event #6013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tinalenguyen wants to merge 1 commit into
base: main
Choose a base branch
from
+26 −1
Open

(core): add function_tools_called event #6013

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions livekit-agents/livekit/agents/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
CloseReason,
ConversationItemAddedEvent,
ErrorEvent,
FunctionToolsCalledEvent,
FunctionToolsExecutedEvent,
MetricsCollectedEvent,
ModelSettings,
Expand Down Expand Up @@ -174,6 +175,7 @@ def __getattr__(name: str) -> typing.Any:
"SpeechCreatedEvent",
"MetricsCollectedEvent",
"SessionUsageUpdatedEvent",
"FunctionToolsCalledEvent",
"FunctionToolsExecutedEvent",
"FunctionCall",
"FunctionCallOutput",
Expand Down
2 changes: 2 additions & 0 deletions livekit-agents/livekit/agents/voice/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
CloseReason,
ConversationItemAddedEvent,
ErrorEvent,
FunctionToolsCalledEvent,
FunctionToolsExecutedEvent,
MetricsCollectedEvent,
RunContext,
Expand Down Expand Up @@ -46,6 +47,7 @@
"CloseReason",
"UserStateChangedEvent",
"AgentStateChangedEvent",
"FunctionToolsCalledEvent",
"FunctionToolsExecutedEvent",
"AgentFalseInterruptionEvent",
"UserTurnExceededEvent",
Expand Down
8 changes: 8 additions & 0 deletions livekit-agents/livekit/agents/voice/events.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,7 @@ def _make_update_pair(
"conversation_item_added",
"agent_false_interruption",
"overlapping_speech",
"function_tools_called",
"function_tools_executed",
"metrics_collected",
"session_usage_updated",
Expand Down Expand Up @@ -363,6 +364,12 @@ class ConversationItemAddedEvent(BaseModel):
created_at: float = Field(default_factory=time.time)


class FunctionToolsCalledEvent(BaseModel):
type: Literal["function_tools_called"] = "function_tools_called"
function_calls: list[FunctionCall]
created_at: float = Field(default_factory=time.time)


class FunctionToolsExecutedEvent(BaseModel):
"""Emitted after a batch of function tools finishes executing.

Expand Down Expand Up @@ -482,6 +489,7 @@ class CloseEvent(BaseModel):
| MetricsCollectedEvent
| SessionUsageUpdatedEvent
| ConversationItemAddedEvent
| FunctionToolsCalledEvent
| FunctionToolsExecutedEvent
| SpeechCreatedEvent
| ErrorEvent
Expand Down
15 changes: 14 additions & 1 deletion livekit-agents/livekit/agents/voice/generation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -568,7 +568,7 @@ async def _execute_tools_task(
"""

from .agent import _set_activity_task_info
from .events import RunContext
from .events import FunctionToolsCalledEvent, RunContext
from .run_result import _MockToolsContextVar

def _tool_completed(out: ToolExecutionOutput) -> None:
Expand All @@ -590,6 +590,7 @@ def _tool_completed(out: ToolExecutionOutput) -> None:
)

tasks: list[asyncio.Task[Any]] = []
called_fnc_calls: list[llm.FunctionCall] = []
try:
async for fnc_call in function_stream:
if tool_choice == "none":
Expand All @@ -602,6 +603,10 @@ def _tool_completed(out: ToolExecutionOutput) -> None:
)
continue

# mirror FunctionToolsExecutedEvent: exclude calls dropped before
# _tool_completed (i.e. tool_choice == "none"), include the rest.
called_fnc_calls.append(fnc_call)

# TODO(theomonnom): assert other tool_choice values

if (function_tool := tool_ctx.function_tools.get(fnc_call.name)) is None:
Expand Down Expand Up @@ -770,6 +775,14 @@ async def _traceable_fnc_tool(
_tool_completed(make_tool_output(fnc_call=fnc_call, output=None, exception=e))
continue

# the model finished emitting its tool calls for this generation; report them
# before their outputs are available (mirrors FunctionToolsExecutedEvent).
if called_fnc_calls:
session.emit(
"function_tools_called",
FunctionToolsCalledEvent(function_calls=called_fnc_calls),
)
Comment on lines +780 to +784

@devin-ai-integration devin-ai-integration Bot Jun 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 FunctionToolsCalledEvent is not forwarded via remote_session transport

The remote_session.py registers handlers for function_tools_executed at livekit-agents/livekit/agents/voice/remote_session.py:374 to forward it over the protobuf transport, but does not register a handler for the new function_tools_called event. This appears intentional since there's no corresponding protobuf message type (AgentSessionEvent.FunctionToolsCalled) defined. If remote consumers (e.g., frontend clients) need to know about tool calls before execution completes, a protobuf message and handler would need to be added in a follow-up.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


await asyncio.shield(asyncio.gather(*tasks, return_exceptions=True))

except asyncio.CancelledError:
Expand Down
Loading