Skip to content

fix(pydantic-ai): Add error handling to agent execution#92

Merged
ericapisani merged 1 commit intomainfrom
ep/investigate-crashing-tests-7d7
Mar 9, 2026
Merged

fix(pydantic-ai): Add error handling to agent execution#92
ericapisani merged 1 commit intomainfrom
ep/investigate-crashing-tests-7d7

Conversation

@ericapisani
Copy link
Member

@ericapisani ericapisani commented Mar 6, 2026

Fix exception handling in pydantic-ai agent template

The pydantic-ai template's agent execution was not handling exceptions
that could occur during agent.run() calls. This caused test execution
to crash instead of gracefully handling errors.

Wrapping the agent execution in a try-except block allows the test to
continue running and properly capture error information when exceptions
occur, making the tests more robust and providing better visibility
into failures.

This fix enables proper validation of error handling behavior in the
pydantic-ai integration tests.

The tests failing within this suite are due to missing fixes in PRs getsentry/sentry-python#5596 and #89

Wrap the agent.run() call in a try-except block to properly handle
exceptions that occur during agent execution. This allows the test to
continue running and capture error information instead of crashing.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions
Copy link

github-actions bot commented Mar 6, 2026

🟡 AI SDK Integration Test Results

Status: 168 tests failing (no regressions)

Summary

Metric main PR Change
Total Tests 384 384
Passed 192 192
Failed 168 168

Test Matrix

Agent Tests

SDK Basic Agent Test Conversation ID Agent Test Long Input Agent Test Tool Call Agent Test Tool Error Agent Test Vision Agent Test
nextjs/mastra
nextjs/vercel blkstr blkstr blkstr blkstr blkstr blkstr
node/manual
node/mastra
node/vercel
php/laravel blkstr blkstr blkstr blkstr blkstr blkstr
python/langgraph as as as as as as
python/manual as as as as as as
python/openai-agents
python/pydantic-ai

Embedding Tests

SDK Basic Embeddings Test
browser/google-genai
browser/langchain
browser/openai
nextjs/google-genai
nextjs/langchain
nextjs/openai
nextjs/vercel
node/google-genai
node/langchain
node/openai
node/vercel
php/laravel
python/google-genai a, blks, blk
python/langchain a, blks, blk
python/litellm a, blks, blk
python/manual a, blks, blk
python/openai a, blks, blk

LLM Tests

SDK Basic Error LLM Test Basic LLM Test Conversation ID LLM Test Long Input LLM Test Multi-Turn LLM Test Vision LLM Test
browser/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
browser/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
browser/langchain blkstr blkstr blkstr blkstr blkstr blkstr
browser/openai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/langchain blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/openai blkstr blkstr blkstr blkstr blkstr blkstr
node/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
node/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
node/langchain blkstr blkstr blkstr blkstr blkstr blkstr
node/manual
node/openai blkstr blkstr blkstr blkstr blkstr blkstr
python/anthropic a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/google-genai a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/langchain a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/litellm a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/manual a, blks, blk a, blks, blk a, blks, blk a, blks, blk a, blks, blk
python/openai a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | str=streaming blk=blocking a=async s=sync


Generated by AI SDK Integration Tests

@ericapisani ericapisani requested a review from a team March 6, 2026 16:30
@ericapisani ericapisani marked this pull request as ready for review March 6, 2026 16:30
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Swallowed exceptions prevent sentry_sdk.capture_exception from firing
    • Added 'raise' statement after printing exception to allow it to propagate to base template's error handler.

Create PR

Or push these changes by commenting:

@cursor push 4a222eb538
Preview (4a222eb538)
diff --git a/src/runner/templates/agents/python/pydantic-ai/template.njk b/src/runner/templates/agents/python/pydantic-ai/template.njk
--- a/src/runner/templates/agents/python/pydantic-ai/template.njk
+++ b/src/runner/templates/agents/python/pydantic-ai/template.njk
@@ -73,6 +73,7 @@
         print(f"Turn {{ loop.index }} Response: {result.output}")
     except Exception as e:
         print(f"Turn {{ loop.index }} Error: {type(e).__name__}: {e}")
+        raise
 {% if not loop.last %}
     print()  # Blank line between turns
 {% endif %}
This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

# Print response
print(f"Turn {{ loop.index }} Response: {result.output}")
except Exception as e:
print(f"Turn {{ loop.index }} Error: {type(e).__name__}: {e}")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swallowed exceptions prevent sentry_sdk.capture_exception from firing

High Severity

The new except Exception as e block catches and prints the error but never re-raises it. When the causeAPIError flag is set, base.python.njk wraps the main() call in its own try-except that calls sentry_sdk.capture_exception(e) — but the exception is now silently swallowed inside main(), so it never propagates. This means Sentry never captures the exception, breaking the error-handling test scenarios that depend on it.

Fix in Cursor Fix in Web

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is entirely accurate as I verified during testing that an error span is being returned from the Python SDK (and not being swallowed as Cursor is saying here) as a result of the generic exception thrown, and is being asserted correctly within the testing framework (example below from a test run):

{
    trace_id: '25a8c19c306a42a2b1bd22b34daa3e84',
    span_id: '85d009fd8743fdb6',
    parent_span_id: '905cfffa695c1f38',
    same_process_as_parent: true,
    op: 'gen_ai.execute_tool',
    description: 'execute_tool read_file',
    start_timestamp: '2026-03-06T16:13:37.239475Z',
    timestamp: '2026-03-06T16:13:37.240453Z',
    origin: 'auto.ai.pydantic_ai',
    status: 'internal_error',
    tags: { status: 'internal_error' },
    data: {
      'thread.id': '8372809984',
      'thread.name': 'MainThread',
      'gen_ai.operation.name': 'execute_tool',
      'gen_ai.tool.type': 'function',
      'gen_ai.tool.name': 'read_file',
      'gen_ai.agent.name': 'file_assistant',
      'gen_ai.tool.input': '{"path": "/nonexistent/file.txt"}'
    }
  }

Copy link
Collaborator

@constantinius constantinius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you!

@ericapisani ericapisani merged commit b1e89cf into main Mar 9, 2026
11 of 12 checks passed
@ericapisani ericapisani deleted the ep/investigate-crashing-tests-7d7 branch March 10, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants