Skip to content

Bug: agent crash when tool execution raise exception due to telemetry #535

@MengJiapeng

Description

@MengJiapeng

Bug Description

The trace_tool_call function in telemetry.py assumes that function_response_event is always provided. However, when a tool execution raises an exception, function_response_event can be None, causing the code to crash with an AttributeError.

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/google/adk/tools/mcp_tool/mcp_session_manager.py", line 163, in wrapper
    return await func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/tools/mcp_tool/mcp_tool.py", line 331, in _run_async_impl
    response = await session.call_tool(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/mcp/client/session.py", line 377, in call_tool
    result = await self.send_request(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/mcp/shared/session.py", line 306, in send_request
    raise McpError(response_or_error.error)
mcp.shared.exceptions.McpError: xxx
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 575, in _execute_single_function_call_async
    function_response_event = await _run_with_trace()
                              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 523, in _run_with_trace
    raise tool_error
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 510, in _run_with_trace
    function_response = await __call_tool_async(
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 936, in __call_tool_async
    return await tool.run_async(args=args, tool_context=tool_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/tools/mcp_tool/mcp_tool.py", line 286, in run_async
    return await super().run_async(args=args, tool_context=tool_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/tools/base_authenticated_tool.py", line 94, in run_async
    return await self._run_async_impl(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/tools/mcp_tool/mcp_session_manager.py", line 176, in wrapper
    return await func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/tools/mcp_tool/mcp_tool.py", line 331, in _run_async_impl
    response = await session.call_tool(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/mcp/client/session.py", line 377, in call_tool
    result = await self.send_request(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/mcp/shared/session.py", line 306, in send_request
    raise McpError(response_or_error.error)
mcp.shared.exceptions.McpError: xxx
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/google/adk/a2a/executor/a2a_agent_executor.py", line 155, in execute
    await self._handle_request(context, event_queue)
  File "/usr/local/lib/python3.12/site-packages/google/adk/a2a/executor/a2a_agent_executor.py", line 225, in _handle_request
    async for adk_event in agen:
  File "/usr/local/lib/python3.12/site-packages/veadk/runner.py", line 134, in wrapper
    async for event in func(
  File "/usr/local/lib/python3.12/site-packages/google/adk/runners.py", line 561, in run_async
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/runners.py", line 549, in _run_with_trace
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/runners.py", line 778, in _exec_with_plugin
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/runners.py", line 538, in execute
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/agents/base_agent.py", line 294, in run_async
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/agents/llm_agent.py", line 468, in _run_async_impl
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 427, in run_async
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 514, in _run_one_step_async
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 712, in _postprocess_async
    async for event in agen:
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 830, in _postprocess_handle_function_calls_async
    if function_response_event := await functions.handle_function_calls_async(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 342, in handle_function_calls_async
    return await handle_function_call_list_async(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 388, in handle_function_call_list_async
    function_response_events = await asyncio.gather(*tasks)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/otel-auto-instrumentation-python/opentelemetry/instrumentation/asyncio/__init__.py", line 299, in trace_coroutine
    return await coro
           ^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/adk/flows/llm_flows/functions.py", line 578, in _execute_single_function_call_async
    trace_tool_call(
  File "/usr/local/lib/python3.12/site-packages/veadk/tracing/telemetry/telemetry.py", line 337, in trace_tool_call
    response: ExtractorResponse = attr_extractor(params)
                                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/veadk/tracing/telemetry/attributes/extractors/tool_attributes_extractors.py", line 129, in tool_gen_ai_tool_output
    function_response = params.function_response_event.get_function_responses()[
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get_function_responses'

Expected Behavior

The tracing code should gracefully handle None values for function_response_event and continue to record available telemetry data (tool name, arguments) without crashing.

Actual Behavior

The code crashes with AttributeError: 'NoneType' object has no attribute 'get_function_responses'

Environment

  • Python version: 3.12+
  • Package: veadk tracing/telemetry module
  • Related to: Tool call tracing and observability

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions