Support OpenAI tool calling (function calling)

README.md

@@ -23,6 +23,7 @@ You sign in once in a browser with your DeepSeek account; your session is saved
 - [Command line](#command-line)
 - [Human-check & proof-of-work (automatic)](#human-check--proof-of-work-automatic)
 - [Models, DeepThink & web search](#models-deepthink--web-search)
+- [Tool calling (function calling)](#tool-calling-function-calling)
 - [Concurrency](#concurrency)
 - [Rate limiting](#rate-limiting)
 - [Project layout](#project-layout)
@@ -220,6 +221,119 @@ on resume. Unknown model names return a `404` (no silent fallback). See
 
 ---
 
+## Tool calling (function calling)
+
+The server accepts OpenAI-style `tools` and returns `tool_calls`, so agent
+frameworks that drive a tool loop work against it:
+
+```python
+resp = client.chat.completions.create(
+    model="deepseek-chat",
+    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
+    tools=[{
+        "type": "function",
+        "function": {
+            "name": "get_weather",
+            "description": "Current weather for a city",
+            "parameters": {
+                "type": "object",
+                "properties": {"city": {"type": "string"}},
+                "required": ["city"],
+            },
+        },
+    }],
+)
+call = resp.choices[0].message.tool_calls[0]   # finish_reason == "tool_calls"
+print(call.function.name, call.function.arguments)
+```
+
+Send the result back as a `tool` message (with the matching `tool_call_id`) and
+the conversation continues as it would against OpenAI.
+
+**This is emulated, not native.** DeepSeek's web chat has no function-calling
+channel, so the bridge puts the tool schemas into the prompt with a strict
+output contract and parses the model's JSON reply back into `tool_calls`. That
+has consequences worth knowing:
+
+- **It's best-effort.** A prompt is not a decoder constraint. Complex or deeply
+  nested parameter schemas are where it frays first.
+- **Unknown tool names in an explicit `{"tool_calls": ...}` envelope are passed
+  through.** Clients with lazy tool loading (a tool-search step, MCP servers
+  resolved on demand) legitimately call tools that aren't in the request's
+  `tools` array, so the call goes to the client to resolve — it reports an
+  unknown tool cleanly, whereas dumping raw JSON at the user does not. The
+  heuristic salvage paths below stay strict: with no envelope to signal intent,
+  a known name is the only thing separating a call from ordinary prose.
+- **Other call dialects are salvaged.** Models don't reliably follow the
+  contract, so replies that narrate the call ReAct-style (`Action: some_tool` /
+  `Action Input: {...}`) or write it as code (`some_tool({"arg": 1})`) are
+  parsed into real calls anyway, including when the model drops a namespace
+  prefix. XML forms (`<some_tool arg="value">`, `<tool_call>...</tool_call>`)
+  are handled too — they turn up when a client's own prompt format leaks into
+  the reply. `deepseek-expert` prefers the code form.
+- **Python dict repr is accepted.** `{'tool_calls': [...]}` with single quotes
+  and `True`/`None` is parsed via `ast.literal_eval` (literals only — it stays a
+  parser, never an evaluator).
+- **Nested wrappers are unwrapped.** A model that names the tool `tool_call` and
+  nests the real call inside `arguments` still gets a working call.
+- **Truncated replies are repaired structurally, never by guessing.** A cut-off
+  object is closed so it can be parsed, but a half-written *value* is dropped
+  with its key: `"tabId": 15` truncated from `1514652929` parses cleanly and
+  points at the wrong thing, and a missing argument that errors is better than a
+  wrong one that gets acted on.
+- **`tool_choice`** supports `"none"`, `"required"`, and pinning a named
+  function.
+- **Streaming buffers.** A tool call is only recognisable once its JSON is
+  complete, so tool-enabled streaming requests emit one delta instead of
+  token-by-token output. Requests without `tools` stream as before.
+
+- **Replies are cut at the first role label.** The prompt serialises the
+  conversation with `User:` / `Assistant:` / `Tool result (...)` labels, and a
+  model will happily keep writing past its own turn — inventing the tool result
+  and the next question. Everything after the first label is dropped, so a
+  fabricated result can never reach the caller as if it were real.
+- **Calls survive a malformed envelope.** If the `{"tool_calls": [...]}` wrapper
+  itself is broken (an array that never closes), the well-formed call objects
+  inside it are still recovered.
+
+Set `DEBUG_REQUESTS=1` to log each request's message roles and offered tool
+names — the quickest way to tell whether a client actually sent `tools`.
+
+Tests: `python -m pytest tests/test_tool_calling.py` (no DeepSeek session
+needed; the upstream client is faked).
+
+---
+
+## Upstream errors
+
+DeepSeek reports failures inside the completion stream as an `event: hint`
+frame, not an HTTP status:
+
+```
+event: hint
+data: {"type":"error","content":"Messages too frequent. Try again later.",
+       "finish_reason":"rate_limit_reached"}
+```
+
+Those frames are surfaced rather than dropped, because a dropped one becomes a
+`200` with empty content — indistinguishable from the model having nothing to
+say, with no reason to show and nothing for a client to back off from.
+
+| Upstream | Response |
+| --- | --- |
+| `rate_limit_reached` | `429` + `Retry-After` (`UPSTREAM_RETRY_AFTER`, default 30) |
+| any other error frame | `502` with the upstream message |
+
+Streaming requests get the same status: the first delta is pulled before the
+response starts, so an error still lands as a status code rather than arriving
+mid-body after a `200`.
+
+Note this is DeepSeek's own per-account limit on the web chat, separate from
+this server's `RATE_LIMIT_PER_MINUTE`. Agent loops are the usual way to hit it —
+each tool round trip is another completion.
+
+---
+
 ## Concurrency
 
 The server bridges a **single** signed-in DeepSeek account behind one shared

deepseek/__init__.py

@@ -1,7 +1,7 @@
 """Unofficial OpenAI-compatible client for chat.deepseek.com."""
 
 from .auth import Session, get_session, login
-from .client import DeepSeekClient, Reply
+from .client import DeepSeekClient, Reply, UpstreamError
 from .pow import DeepSeekPow
 
-__all__ = ["Session", "get_session", "login", "DeepSeekClient", "Reply", "DeepSeekPow"]
+__all__ = ["Session", "get_session", "login", "DeepSeekClient", "Reply", "UpstreamError", "DeepSeekPow"]

deepseek/client.py

@@ -44,6 +44,25 @@
 _CID_SEP = ":"
 
 
+class UpstreamError(RuntimeError):
+    """An error DeepSeek reported inside the completion stream.
+
+    The stream carries failures as an `event: hint` frame rather than an HTTP
+    status, e.g. {"type": "error", "content": "Messages too frequent. Try again
+    later.", "finish_reason": "rate_limit_reached"}. Dropping those frames turns
+    a rate limit into a silent empty reply, which the caller can only report as
+    "the model returned nothing" -- no reason to show, nothing to back off from.
+    """
+
+    def __init__(self, message: str, finish_reason: str = ""):
+        super().__init__(message or "DeepSeek reported an error")
+        self.finish_reason = finish_reason or ""
+
+    @property
+    def is_rate_limit(self) -> bool:
+        return "rate_limit" in self.finish_reason.lower()
+
+
 def _encode_cid(session_id: str, message_id: Optional[int]) -> str:
     if message_id is None:
         return session_id
@@ -259,6 +278,10 @@ def _parse_sse(lines, meta: Optional[dict] = None) -> Iterator[str]:
         except json.JSONDecodeError:
             continue
 
+        if obj.get("type") == "error" or obj.get("finish_reason") == "rate_limit_reached":
+            raise UpstreamError(str(obj.get("content") or "").strip(),
+                                str(obj.get("finish_reason") or ""))
+
         v = obj.get("v")
 
         # Snapshot frame: full response object.

server/api.py

@@ -21,6 +21,9 @@
 
 from __future__ import annotations
 
+import json
+import logging
+import os
 import threading
 import time
 
@@ -30,7 +33,7 @@
 from starlette.concurrency import run_in_threadpool
 
 from deepseek.auth import LoginRequired
-from deepseek.client import DeepSeekClient
+from deepseek.client import DeepSeekClient, UpstreamError
 
 from .config import (
     MODEL_MAP,
@@ -39,12 +42,30 @@
     is_known_model,
     resolve_model_type,
 )
-from .openai_format import completion_response, messages_to_prompt, stream_chunks
+from .openai_format import (
+    completion_response,
+    extract_tool_calls,
+    trim_at_role_boundary,
+    messages_to_prompt,
+    stream_chunks,
+    stream_chunks_with_tools,
+)
 from .ratelimit import RateLimiter, install_rate_limit
 from .schemas import ChatCompletionRequest
 
 load_dotenv()
 
+# uvicorn configures logging with disable_existing_loggers, so hang our
+# debug output off its own logger to be sure it reaches the console.
+log = logging.getLogger("uvicorn.error")
+# Set DEBUG_REQUESTS=1 to log each request's shape (roles + tool names). Useful
+# when a client's tool calls aren't arriving: it shows whether `tools` was sent
+# at all, which is the difference between "bridge ignored them" and "client
+# never offered them".
+DEBUG_REQUESTS = os.getenv("DEBUG_REQUESTS", "").lower() in ("1", "true", "yes", "on")
+# Seconds to advertise in Retry-After when DeepSeek rate-limits us.
+UPSTREAM_RETRY_AFTER = os.getenv("UPSTREAM_RETRY_AFTER", "30")
+
 app = FastAPI(title="DeepSeek OpenAI-compatible API", version="0.1.0")
 install_rate_limit(app, RateLimiter(limit=RATE_LIMIT_PER_MINUTE, window=60.0))
 
@@ -73,13 +94,53 @@ def get_client() -> DeepSeekClient:
     return _client
 
 
-def _error(message: str, status: int = 500, err_type: str = "server_error"):
+def _error(message: str, status: int = 500, err_type: str = "server_error",
+           headers: dict = None):
     return JSONResponse(
         status_code=status,
         content={"error": {"message": message, "type": err_type}},
+        headers=headers or None,
     )
 
 
+def _upstream_error(exc: UpstreamError):
+    """Map a DeepSeek stream error onto the status a client can act on.
+
+    A rate limit must be a 429 with Retry-After: OpenAI-compatible clients back
+    off and retry on that, whereas a 200 with empty content just looks like the
+    model had nothing to say.
+    """
+    if exc.is_rate_limit:
+        return _error(str(exc), status=429, err_type="rate_limit_exceeded",
+                      headers={"Retry-After": UPSTREAM_RETRY_AFTER})
+    return _error(f"DeepSeek reported: {exc}", status=502, err_type="upstream_error")
+
+
+class _Prefetched:
+    """A stream with its first delta already pulled.
+
+    The endpoint consumes one delta before returning a response, so an error
+    frame -- which DeepSeek sends immediately -- becomes a real status code
+    instead of a 200 whose body turns out to be an error mid-flight.
+    """
+
+    def __init__(self, stream, iterator, first):
+        # `iterator` is the ALREADY-ADVANCED iterator, not the stream object.
+        # Re-iterating the stream replays it from the beginning, which silently
+        # duplicates the prefetched delta ("II'll..." instead of "I'll...") and
+        # can corrupt a tool call whose reply starts with `{`.
+        self._stream, self._iter, self._first = stream, iterator, first
+
+    def __iter__(self):
+        if self._first is not None:
+            yield self._first
+        yield from self._iter
+
+    @property
+    def conversation_id(self):
+        return getattr(self._stream, "conversation_id", None)
+
+
 @app.get("/healthz")
 def healthz():
     return {"status": "ok"}
@@ -112,7 +173,16 @@ async def chat_completions(req: ChatCompletionRequest):
     # A thread's model is fixed when it's created, so on resume we ignore `model`
     # (the OpenAI SDK always sends one) and let the existing thread's model stand.
     model_type = None if req.conversation_id else resolve_model_type(req.model)
-    prompt = messages_to_prompt(req.messages)
+    if DEBUG_REQUESTS:
+        log.warning("request: model=%s stream=%s roles=%s tools=%s tool_choice=%s",
+                    req.model, req.stream, [m.role for m in req.messages],
+                    [ (t.get("function") or t).get("name") for t in (req.tools or []) ],
+                    req.tool_choice)
+
+    prompt = messages_to_prompt(req.messages, req.tools, req.tool_choice)
+    # DeepSeek has no native tool channel; `tools` are emulated in the prompt
+    # and parsed back out of the reply. `tool_choice: none` means don't offer.
+    tools = None if req.tool_choice == "none" else req.tools
 
     try:
         # Off the event loop: get_client() uses Playwright's sync API, which
@@ -124,12 +194,24 @@ async def chat_completions(req: ChatCompletionRequest):
         return _error(f"Failed to initialise DeepSeek session: {e}")
 
     if req.stream:
+        raw = client.stream(
+            prompt, conversation_id=req.conversation_id,
+            model=model_type, thinking=req.thinking, search=req.search,
+        )
+        it = iter(raw)
+        try:
+            first = await run_in_threadpool(lambda: next(it, None))
+        except UpstreamError as e:
+            return _upstream_error(e)
+        except Exception as e:
+            return _error(f"DeepSeek request failed: {e}")
+
         def gen():
-            stream = client.stream(
-                prompt, conversation_id=req.conversation_id,
-                model=model_type, thinking=req.thinking, search=req.search,
-            )
-            yield from stream_chunks(req.model, stream)
+            stream = _Prefetched(raw, it, first)
+            if tools:
+                yield from stream_chunks_with_tools(req.model, stream, tools)
+            else:
+                yield from stream_chunks(req.model, stream)
 
         return StreamingResponse(gen(), media_type="text/event-stream")
 
@@ -138,7 +220,19 @@ def gen():
             client.chat, prompt, req.conversation_id,
             model_type, req.thinking, req.search,
         )
+    except UpstreamError as e:
+        return _upstream_error(e)
     except Exception as e:
         return _error(f"DeepSeek request failed: {e}")
 
-    return completion_response(req.model, reply.text, prompt, reply.conversation_id)
+    calls, text = extract_tool_calls(reply.text, tools)
+    if not tools:
+        # No tools this turn, so extract_tool_calls returned early — the reply
+        # can still run on past its turn, so trim it here too.
+        text = trim_at_role_boundary(text)
+    if DEBUG_REQUESTS:
+        log.warning("reply: calls=%s text=%r",
+                    [(c["function"]["name"], c["function"]["arguments"]) for c in calls or []],
+                    (text or "")[:200])
+    return completion_response(req.model, text, prompt, reply.conversation_id,
+                               tool_calls=calls)