Repository navigation
Python: Report the finish reason and keep zero-prompt-token usage for Ollama chat - #14674
alexcastgonz (pcbeingused333) wants to merge 2 commits into
Conversation
… Ollama chat Ollama reports why generation stopped in done_reason, but the connector never passed it on, so a reply cut off by the token limit looked like a complete one. Map stop/length (and stop with tool calls) to FinishReason on the four response paths, and keep the usage when the prompt was fully cached (prompt_eval_count 0). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Both streaming converters incorrectly report STOP instead of TOOL_CALLS when tool calls precede an empty terminal chunk, requiring a fix and regression coverage for both response types.
1 open issue
What changed in this PR
This PR updates the Python Ollama chat connector to expose generation finish reasons and preserve usage data when prompt token counts are zero.
Changes:
- Maps Ollama’s
done_reasonto finish reasons for typed and mapping responses, including streaming. - Preserves token usage when counts are zero rather than missing.
- Adds tests for finish reasons, terminal streaming chunks, and zero-prompt-token usage.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/ollama/services/test_ollama_chat_completion.py |
Adds finish-reason and zero-token usage tests; updates response doubles. |
python/semantic_kernel/connectors/ai/ollama/services/ollama_chat_completion.py |
Adds finish-reason mapping across response paths and retains zero-token usage. |
Review effort: Balanced
| inner_content=response, | ||
| ai_model_id=self.ai_model_id, | ||
| metadata=self._get_metadata_from_chat_response(response), | ||
| finish_reason=self._get_finish_reason(response.done_reason, bool(response.message.tool_calls)), |
There was a problem hiding this comment.
Confirmed on the current head using the actual mapping converter. A chunk containing tool_calls=[{"function":{"name":"fn","arguments":{"x":1}}}], followed by {"message":{"content":""},"done_reason":"stop"} with tool_call_index=1, produces finish reasons None then STOP. Adding the two StreamingChatMessageContent objects keeps the function call but reports STOP, not TOOL_CALLS. Both streaming converters need to account for tool calls from earlier chunks; please add this two-chunk regression case.
…tream chunk The streaming converters only looked at the current chunk's tool calls, so an empty terminal chunk with done_reason=stop after tool-call chunks was reported as STOP. Also consider tool calls already seen in the stream.
|
Thanks, confirmed. Both streaming converters now treat This comment was drafted with Claude Code. |
PRABHU KIRAN VANDRANKI (VANDRANKI)
left a comment
There was a problem hiding this comment.
I ran this on current main (df1cb33) and on main with this diff applied (python venv, ollama client 0.6.3). I did not mock the client. I started a local HTTP server that speaks the Ollama /api/chat format (plain JSON and NDJSON streaming) and pointed OllamaChatCompletion(host=...) at it, so the real ollama.AsyncClient parsed the responses. I do not have a real Ollama server.
Main reproduces both problems. Non-streaming get_chat_message_contents:
| server reply | main finish_reason | PR finish_reason |
|---|---|---|
done_reason: stop |
None | STOP |
done_reason: length |
None | LENGTH |
stop plus tool call |
None | TOOL_CALLS |
done_reason: load |
None | None |
no done_reason |
None | None |
Usage with prompt_eval_count: 0 and eval_count: 5: main has no usage in the metadata, the PR gives (0, 5). With both counts 0 the PR gives (0, 0). With both counts missing it gives no usage, as on main.
Streaming (get_streaming_chat_message_contents), finish reason per chunk:
- text, text, then
stop: main[None, None, None], PR[None, None, STOP] - text, then
length: PR[None, LENGTH] - a tool call chunk, then an empty terminal
stopchunk: PR[None, TOOL_CALLS]. This is the case the Copilot review raised, and it works on the current head. - a tool call in the terminal chunk itself: PR
[None, TOOL_CALLS] - terminal chunk with
done_reason: load: PR[None, None]
Auto function calling still works. I ran a Time.get_time plugin with FunctionChoiceBehavior.Auto() against the fake server, non-streaming and streaming. Both runs made the tool call, invoked it, and returned "It is 12:00." with the same history roles (user, assistant, tool) on main and the PR. The streamed run reports TOOL_CALLS on the first turn and STOP on the last on the PR. On main all chunks report None.
Tests. tests/unit/connectors/ai/ollama: 62 passed on main, 75 passed on the PR (13 new cases). If I copy the PR's test file onto main, 10 of those fail on main, including the zero-prompt-token usage test, which fails with KeyError: 'usage'.
Limits. The Mapping response paths (_create_chat_message_content and _create_streaming_chat_message_content) were run only through the PR's own unit tests, since the real client returns ChatResponse objects. The package pin is ollama ~= 0.4. I looked at the source of the 0.4.0 wheel and done_reason is a field on the response there (read only, I did not run 0.4.0).
Approving.

Motivation and Context
Ollama reports why generation stopped in
done_reason, but the Ollama chat connector never put it on the returned content, sofinish_reasonwas alwaysNone. A reply cut off by the token limit (done_reason == "length") could not be told apart from a complete one. Separately,_get_metadata_from_chat_responsedropped the usage wheneverprompt_eval_countwas 0 (a fully cached prompt), because it tested truthiness instead ofis not None.Description
OllamaChatCompletion._get_finish_reasonmapsdone_reason:length->FinishReason.LENGTH;stop->FinishReason.STOP, orFinishReason.TOOL_CALLSwhen the message has tool calls (Ollama reportsstopfor those, other connectors reporttool_calls); anything else (including a streaming chunk that is not the last one) ->None.ChatResponseand mapping responses, streaming and not.ChatResponseusage check now usesis not None, soprompt_eval_count == 0keeps the usage.tests/unit/connectors/ai/ollama/services/test_ollama_chat_completion.py): finish reason for stop / length / stop with tool calls / unknown / missing on realollamaChatResponseobjects and on mappings; a streamed response where only the last chunk carrieslength; usage withprompt_eval_count == 0. These fail onmainand pass with the change; the full Ollama unit test directory passes andruff check/ruff format --checkare clean. Not run against a live Ollama server.Contribution Checklist
This change was made with Claude Code (Claude Sonnet 5.5).