Skip to content

Conversation

@anakin87
Copy link
Member

@anakin87 anakin87 commented Jan 20, 2026

Related Issues

Proposed Changes:

  • expand ToolCallResult.result to not only accepts str but also list[TextContent|ImageContent]
  • modify OpenAIResponsesChatGenerator to handle this. (Haven't implemented in OpenAIChatGenerator since the Chat Completions API only accept str in Tool results).
  • add outputs_to_result initialization parameter to Tool (similar to existing parameters), to allow returning the raw Tool result (instead of stringifying it) and optionally transforming it
  • add the same parameter to ComponentTool and PipelineTool
  • refactor ToolInvoker to support this feature

How did you test it?

CI, added several tests

A simple end-to-end example can also be found in tests:

def test_live_run_agent_with_images_in_tool_result(self, test_files_path):

Notes for the reviewer

Opened this PR to get feedback. Something is still missing.

  • Tested with Anthropic: wip: Anthropic support imgs tool results haystack-core-integrations#2769. Bedrock should be similar. I am also confident that the current abstractions can work with Gemini and other providers when they support this feature.
  • Missing:
    • release note
    • support new ToolCallResult in Jinja2 ChatMessage extension - not needed but in general we tried to keep feature parity between the ChatMessage dataclass and the extension
    • new small section in documentation

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@vercel
Copy link

vercel bot commented Jan 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
haystack-docs Ready Ready Preview, Comment Jan 21, 2026 2:59pm

Request Review

@github-actions github-actions bot added topic:tests type:documentation Improvements on the docs labels Jan 20, 2026
@coveralls
Copy link
Collaborator

coveralls commented Jan 21, 2026

Pull Request Test Coverage Report for Build 21214269984

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 32 unchanged lines in 6 files lost coverage.
  • Overall coverage increased (+0.1%) to 92.394%

Files with Coverage Reduction New Missed Lines %
components/generators/chat/openai_responses.py 2 87.38%
dataclasses/chat_message.py 3 99.08%
tools/pipeline_tool.py 5 85.0%
tools/tool.py 6 95.8%
tools/component_tool.py 7 92.78%
components/tools/tool_invoker.py 9 96.45%
Totals Coverage Status
Change from base Build 21212784205: 0.1%
Covered Lines: 14492
Relevant Lines: 15685

💛 - Coveralls

@anakin87 anakin87 changed the title wip: support images in tool results feat: support passing images in Tool results Jan 21, 2026
@anakin87 anakin87 marked this pull request as ready for review January 21, 2026 15:29
@anakin87 anakin87 requested a review from a team as a code owner January 21, 2026 15:29
@anakin87 anakin87 requested review from julian-risch and sjrl and removed request for a team and julian-risch January 21, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants