fix(llm): handle LiteLLM supports_response_schema introspection failure gracefully (#6412)

[Hasnaathussain]

Pull Request Description

Description

Closes #6412

This PR resolves an issue in LLM._validate_call_params() where unhandled LiteLLM introspection failures in supports_response_schema() would cause _validate_call_params() to crash or skip parameter validation.

Root Cause

In crewai/llm.py, _validate_call_params() invoked supports_response_schema() directly without exception handling. If LiteLLM failed to introspect an unmapped/custom model ID or raised an exception, _validate_call_params() crashed unhandled.

Proposed Changes

1. Added exception handling around supports_response_schema() in _validate_call_params(), mirroring the pattern used in supports_function_calling().
2. On LiteLLM introspection errors, the failure is logged at DEBUG level and is_supported defaults to True (permitting modern models and proxies with custom model names to execute).
3. Added unit test test_validate_call_params_handles_introspection_error() in lib/crewai/tests/test_llm.py.

Testing

• Verified test_validate_call_params_handles_introspection_error passes.
• Verified linting and formatting with ruff check and ruff format.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 395443be-f083-4a91-97c7-3aee5aaa0d8d

📥 Commits

Reviewing files that changed from the base of the PR and between 8ac9f55 and 33a57b8.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/test_llm.py

🚧 Files skipped from review as they are similar to previous changes (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/test_llm.py

---

📝 Walkthrough

Walkthrough

_validate_call_params now applies static provider capability lists, skips unnecessary response-format introspection, handles unavailable or failing LiteLLM checks, and raises only for confirmed unsupported formats. Tests cover these validation paths and reasoning-effort logging.

Changes

Provider capability validation

Layer / File(s)	Summary
Static capability checks and validation coverage lib/crewai/src/crewai/llm.py, lib/crewai/tests/test_llm.py	Static provider lists guide reasoning_effort and response_format validation. LiteLLM checks are conditional, introspection failures are logged and treated as supported, and confirmed unsupported formats raise ValueError; tests cover these behaviors.

Sequence Diagram(s)

sequenceDiagram
  participant LLM
  participant LiteLLM
  participant Logger
  LLM->>LLM: Check static provider capabilities
  LLM->>LiteLLM: Inspect response-schema support when required
  LiteLLM-->>LLM: Return support status or raise an exception
  LLM->>Logger: Log unsupported reasoning_effort or introspection failure
  LLM-->>LLM: Raise ValueError only for confirmed unsupported response_format

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the LiteLLM introspection failure fix in LLM validation.
Description check	✅ Passed	The description matches the implemented change and its tests.
Linked Issues check	✅ Passed	The PR adds static capability checks and graceful introspection fallback, satisfying #6412's requirements.
Out of Scope Changes check	✅ Passed	The changes stay focused on llm validation logic and unit tests, with no obvious unrelated additions.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/tests/test_llm.py`:
- Around line 1205-1217: Update
test_validate_call_params_skips_check_when_no_response_format so it uses only
parameters that require no validation, rather than reasoning_effort. Add a
separate test covering reasoning_effort with supported and unsupported
model/provider capabilities, verifying the expected fallback behavior while
preserving response_format validation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 33b040d2-b6ff-43b2-8fc9-0acc0e85c7c6

📥 Commits

Reviewing files that changed from the base of the PR and between 5c61444 and 282e438.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/test_llm.py

🚧 Files skipped from review as they are similar to previous changes (1)

• lib/crewai/src/crewai/llm.py

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/tests/test_llm.py`:
- Around line 1185-1196: Update
test_validate_call_params_handles_introspection_error to accept caplog, invoke
_validate_call_params within the existing patched introspection failure, and
assert the DEBUG logs contain “LiteLLM supports_response_schema check failed”
while preserving the no-exception fallback assertion.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 488e39e5-3b1b-46f2-82fc-d182853081e3

📥 Commits

Reviewing files that changed from the base of the PR and between 282e438 and 5781925.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/test_llm.py

🚧 Files skipped from review as they are similar to previous changes (1)

• lib/crewai/src/crewai/llm.py