Align tool context surface with Python (Tool.id, ProviderTool abstract class, Toolset symbol)

[toubatbrian]

Summary

Closes the remaining parity gaps between this repo's agents/src/llm/tool_context.ts and Python's livekit-agents/livekit/agents/llm/tool_context.py. Three changes, all behind the same changeset:

1. Tool.id on the base interface

Python's abstract Tool declares id: str. JS now does the same. For function tools id === name; for provider tools it's the provider tool id. ToolContext keys the function-tool map and runs equality on tool.id, mirroring Python's _fnc_tools_map[tool.info.name].

2. ProviderTool is now an abstract class

Python's ProviderTool is a class plugins extend (OpenAITool(ProviderTool, ABC) → WebSearch(OpenAITool), with each leaf adding fields and a config serializer). The JS shape didn't support this — ProviderDefinedTool was an interface carrying an unused config: Record<string, unknown>, and the tool({ id, config }) factory hid the construction path.

Now:

• Renamed ProviderDefinedTool → ProviderTool, isProviderDefinedTool → isProviderTool.
• Converted to abstract class ProviderTool implements Tool with readonly symbol fields so subclasses inherit the marker.
• Removed the config field and the tool({ id }) factory overload.
• ToolType literal 'provider-defined' → 'provider'.

Plugin authoring follows Python 1:1:

// plugins/openai/src/tools.ts (illustrative)
export abstract class OpenAITool extends llm.ProviderTool {
  abstract get config(): Record<string, unknown>;
}

export class WebSearch extends OpenAITool {
  constructor(public readonly filters?: string) {
    super({ id: 'openai_web_search' });
  }
  get config() {
    return { type: 'web_search', filters: this.filters };
  }
}

new WebSearch('safe') passes isProviderTool and lands in ToolContext._providerTools. The TODO to actually wire each plugin's config getter through its LLM serializer is tracked as AJS-112 and unchanged by this PR.

3. Symbol-based detection for all three tool kinds

FunctionTool and ProviderTool already had symbol markers. Toolset was the odd one out (instanceof Toolset). Added TOOLSET_SYMBOL + an isToolset() guard, and swapped the one instanceof Toolset in agent_activity.ts.

BREAKING

See .changeset/list-syntax-toolcontext.md for the rendered release note. TL;DR for callers:

• Replace ProviderDefinedTool / isProviderDefinedTool with ProviderTool / isProviderTool.
• Replace tool({ id, config: {} }) with new MyProviderTool(...) (a ProviderTool subclass).
• Replace tool.type === 'provider-defined' with tool.type === 'provider'.

Test plan

• pnpm test agents/src/llm — 312/312 pass (2 pre-existing skips)
• pnpm build:agents and pnpm build:plugins — all 31 publishable packages build clean
• Added test asserting ProviderTool is rejected for direct instantiation (@ts-expect-error) and a subclass works
• Existing ToolContext parity tests (function tools / provider tools / toolsets / equals / flatten / hasTool) updated to use new ProviderTool subclasses

[changeset-bot]

🦋 Changeset detected

Latest commit: d05b896

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 31 packages

Name	Type
@livekit/agents-plugin-google	Major
@livekit/agents	Major
@livekit/agents-plugin-anam	Major
@livekit/agents-plugin-assemblyai	Major
@livekit/agents-plugin-baseten	Major
@livekit/agents-plugin-bey	Major
@livekit/agents-plugin-cartesia	Major
@livekit/agents-plugin-cerebras	Major
@livekit/agents-plugin-deepgram	Major
@livekit/agents-plugin-elevenlabs	Major
@livekit/agents-plugin-fishaudio	Major
@livekit/agents-plugin-hedra	Major
@livekit/agents-plugin-hume	Major
@livekit/agents-plugin-inworld	Major
@livekit/agents-plugin-lemonslice	Major
@livekit/agents-plugin-liveavatar	Major
@livekit/agents-plugin-livekit	Major
@livekit/agents-plugin-minimax	Major
@livekit/agents-plugin-mistral	Major
@livekit/agents-plugin-mistralai	Major
@livekit/agents-plugin-neuphonic	Major
@livekit/agents-plugin-openai	Major
@livekit/agents-plugin-phonic	Major
@livekit/agents-plugin-resemble	Major
@livekit/agents-plugin-rime	Major
@livekit/agents-plugin-runway	Major
@livekit/agents-plugin-sarvam	Major
@livekit/agents-plugin-silero	Major
@livekit/agents-plugins-test	Major
@livekit/agents-plugin-trugen	Major
@livekit/agents-plugin-xai	Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b883bfb56

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Validate execute before constructing function tool

The new tool() implementation always returns a function tool once name passes validation, but it no longer checks that execute is present and callable. In JavaScript consumers (or any-typed TS call sites), tool({ name, description }) now succeeds and stores execute: undefined, which only fails later when the tool is invoked instead of failing fast during registration. The previous behavior rejected non-function/non-provider shapes early, so this is a regression in runtime safety.

Useful? React with 👍 / 👎.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 8 additional findings in Devin Review.

[devin-ai-integration]

🔴 toToolsConfig error check fires before onlySingleType guard, breaking LLM path when both function tools and geminiTools are configured

The error check at plugins/google/src/utils.ts:204 (if (tools.length > 0 && providerTools.length > 0) throw ...) runs before the onlySingleType early-return at line 208. This means when the Google LLM calls toToolsConfig({ ..., onlySingleType: true }) (plugins/google/src/llm.ts:358-362), and the user has both function tools and geminiTools (e.g. Google Search), the function throws instead of gracefully returning only the function tools.

Additionally, the same error causes a regression in the realtime session path (plugins/google/src/beta/realtime/realtime_api.ts:1416-1420), which does NOT set onlySingleType. The old realtime code merged functionDeclarations and geminiTools into a single types.Tool object ({ functionDeclarations: [...], ...geminiTools }), which the Gemini Live API supports. The new code separates them and throws an error, breaking users who combine function tools with Google Search or other Gemini-provided tools.

Old realtime code that worked with mixed tools

tools: [
  {
    functionDeclarations: this.geminiDeclarations,
    ...this.options.geminiTools,  // e.g. { googleSearch: {} }
  },
]

Prompt for agents

The error check on line 204 that throws when both function tools and provider tools exist is (a) too early -- it should run AFTER the onlySingleType early-return at line 208 so the LLM path can gracefully return just function tools, and (b) too strict for the realtime path -- the old realtime code merged functionDeclarations and geminiTools into a single types.Tool object which the Gemini API supports.

To fix:
1. Move the onlySingleType check (line 208-209) BEFORE the error check (line 204-206).
2. For the realtime path (onlySingleType=false), instead of throwing, merge functionDeclarations and provider tool configs into a single types.Tool object like the old code did. This means combining { functionDeclarations: [...] } with e.g. { googleSearch: {} } into one object: { functionDeclarations: [...], googleSearch: {} }.

Affected files: plugins/google/src/utils.ts (toToolsConfig function, lines 163-215).

---

Was this helpful? React with 👍 or 👎 to provide feedback.