Skip to content

Python: Report the finish reason and keep zero-prompt-token usage for Ollama chat - #14674

Open
alexcastgonz (pcbeingused333) wants to merge 2 commits into
microsoft:mainfrom
pcbeingused333:fix/ollama-finish-reason
Open

alexcastgonz (pcbeingused333) wants to merge 2 commits into
microsoft:mainfrom
pcbeingused333:fix/ollama-finish-reason

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Motivation and Context

Ollama reports why generation stopped in done_reason, but the Ollama chat connector never put it on the returned content, so finish_reason was always None. A reply cut off by the token limit (done_reason == "length") could not be told apart from a complete one. Separately, _get_metadata_from_chat_response dropped the usage whenever prompt_eval_count was 0 (a fully cached prompt), because it tested truthiness instead of is not None.

Description

  • OllamaChatCompletion._get_finish_reason maps done_reason: length -> FinishReason.LENGTH; stop -> FinishReason.STOP, or FinishReason.TOOL_CALLS when the message has tool calls (Ollama reports stop for those, other connectors report tool_calls); anything else (including a streaming chunk that is not the last one) -> None.
  • It is applied on all four paths: ChatResponse and mapping responses, streaming and not.
  • The ChatResponse usage check now uses is not None, so prompt_eval_count == 0 keeps the usage.
  • Tests (tests/unit/connectors/ai/ollama/services/test_ollama_chat_completion.py): finish reason for stop / length / stop with tool calls / unknown / missing on real ollama ChatResponse objects and on mappings; a streamed response where only the last chunk carries length; usage with prompt_eval_count == 0. These fail on main and pass with the change; the full Ollama unit test directory passes and ruff check / ruff format --check are clean. Not run against a live Ollama server.

Contribution Checklist

This change was made with Claude Code (Claude Sonnet 5.5).

… Ollama chat

Ollama reports why generation stopped in done_reason, but the connector never
passed it on, so a reply cut off by the token limit looked like a complete one.
Map stop/length (and stop with tool calls) to FinishReason on the four response
paths, and keep the usage when the prompt was fully cached (prompt_eval_count 0).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 01:52
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Oct 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Both streaming converters incorrectly report STOP instead of TOOL_CALLS when tool calls precede an empty terminal chunk, requiring a fix and regression coverage for both response types.

1 open issue
What changed in this PR

This PR updates the Python Ollama chat connector to expose generation finish reasons and preserve usage data when prompt token counts are zero.

Changes:

  • Maps Ollama’s done_reason to finish reasons for typed and mapping responses, including streaming.
  • Preserves token usage when counts are zero rather than missing.
  • Adds tests for finish reasons, terminal streaming chunks, and zero-prompt-token usage.
File Description
python/​tests/​unit/​connectors/​ai/​ollama/​services/​test_ollama_chat_completion.py Adds finish-reason and zero-token usage tests; updates response doubles.
python/​semantic_kernel/​connectors/​ai/​ollama/​services/​ollama_chat_completion.py Adds finish-reason mapping across response paths and retains zero-token usage.

Review effort: Balanced

inner_content=response,
ai_model_id=self.ai_model_id,
metadata=self._get_metadata_from_chat_response(response),
finish_reason=self._get_finish_reason(response.done_reason, bool(response.message.tool_calls)),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed on the current head using the actual mapping converter. A chunk containing tool_calls=[{"function":{"name":"fn","arguments":{"x":1}}}], followed by {"message":{"content":""},"done_reason":"stop"} with tool_call_index=1, produces finish reasons None then STOP. Adding the two StreamingChatMessageContent objects keeps the function call but reports STOP, not TOOL_CALLS. Both streaming converters need to account for tool calls from earlier chunks; please add this two-chunk regression case.

…tream chunk

The streaming converters only looked at the current chunk's tool calls, so
an empty terminal chunk with done_reason=stop after tool-call chunks was
reported as STOP. Also consider tool calls already seen in the stream.
@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Thanks, confirmed. Both streaming converters now treat tool_call_index > 0 (tool calls already emitted earlier in the stream) like the current chunk's own tool calls, so an empty terminal chunk with done_reason="stop" reports TOOL_CALLS. Added a two-chunk regression test for each response type (typed ChatResponse and mapping); both fail without the change and the Ollama unit suite passes (8937fa5).

This comment was drafted with Claude Code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran this on current main (df1cb33) and on main with this diff applied (python venv, ollama client 0.6.3). I did not mock the client. I started a local HTTP server that speaks the Ollama /api/chat format (plain JSON and NDJSON streaming) and pointed OllamaChatCompletion(host=...) at it, so the real ollama.AsyncClient parsed the responses. I do not have a real Ollama server.

Main reproduces both problems. Non-streaming get_chat_message_contents:

server reply main finish_reason PR finish_reason
done_reason: stop None STOP
done_reason: length None LENGTH
stop plus tool call None TOOL_CALLS
done_reason: load None None
no done_reason None None

Usage with prompt_eval_count: 0 and eval_count: 5: main has no usage in the metadata, the PR gives (0, 5). With both counts 0 the PR gives (0, 0). With both counts missing it gives no usage, as on main.

Streaming (get_streaming_chat_message_contents), finish reason per chunk:

  • text, text, then stop: main [None, None, None], PR [None, None, STOP]
  • text, then length: PR [None, LENGTH]
  • a tool call chunk, then an empty terminal stop chunk: PR [None, TOOL_CALLS]. This is the case the Copilot review raised, and it works on the current head.
  • a tool call in the terminal chunk itself: PR [None, TOOL_CALLS]
  • terminal chunk with done_reason: load: PR [None, None]

Auto function calling still works. I ran a Time.get_time plugin with FunctionChoiceBehavior.Auto() against the fake server, non-streaming and streaming. Both runs made the tool call, invoked it, and returned "It is 12:00." with the same history roles (user, assistant, tool) on main and the PR. The streamed run reports TOOL_CALLS on the first turn and STOP on the last on the PR. On main all chunks report None.

Tests. tests/unit/connectors/ai/ollama: 62 passed on main, 75 passed on the PR (13 new cases). If I copy the PR's test file onto main, 10 of those fail on main, including the zero-prompt-token usage test, which fails with KeyError: 'usage'.

Limits. The Mapping response paths (_create_chat_message_content and _create_streaming_chat_message_content) were run only through the PR's own unit tests, since the real client returns ChatResponse objects. The package pin is ollama ~= 0.4. I looked at the source of the 0.4.0 wheel and done_reason is a field on the response there (read only, I did not run 0.4.0).

Approving.

This branch was previously deployed

1 inactive deployment
github-app-auth — 8937fa50 Deployed Nov 2, 2026 by pcbeingused333 via add_label #29536
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants