From 41bb3358179154353e8aed7ae58cb0405f2df5ac Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> Date: Mon, 21 Sep 2026 06:05:34 +0000 Subject: [PATCH] docs: design of record for the local engine API Three products are written against a local engine endpoint that has never existed. redrob-browser POSTs to /experimental/external/chat/completions and streams the reply; no such route is served, and the string "external" does not appear under packages/redrob/src/server. redrob-reblend ranks a LOCAL transport its own sidecar cannot use. redrob-office compiles this engine at release time, ships it inside its installer behind a packaging guard, and never calls it. The products that drive the engine successfully drive its session API. The ones that tried to give it an OpenAI face are the broken ones. So this serves that face for real, at one stable path. Decisions recorded here: - POST /v1/chat/completions is the core path, with no alias. It sits on packages/llm and the provider registry, not on Session: a completion is stateless and a session is not, and local inference then falls out as just another resolved provider. - Tool ownership is decided by the request, not configured. `tools` present means the CALLER owns them and the engine returns tool_calls; absent means the engine runs its own. Office edits a document held in its renderer and browser drives a live tab, so for them caller ownership is the only correct answer. - Inbound auth is the server's existing Basic. Outbound is the engine's OWN Console credential, which is what makes one login serve every product. - 401 carries code "engine_not_authenticated", and that code alone may trigger a sign-in prompt. Branching on message text instead is how a network timeout shipped as "please log in". - Every /experimental route is promoted in one breaking rename. No product calls any of them; the only callers are this repo's generated SDKs, which httpapi-codegen regenerates. No compatibility window is owed to callers that do not exist. - Products stop bundling the binary. One shared daemon, recorded at ~/.redrob/run/server.json (0600), resolved via REDROB_CODE_BIN then $HOME/.redrob/bin then PATH. Per-product processes were rejected: with local inference that loads the same weights N times. - Nothing pins an engine version. /capabilities gains a chatCompletions entry and products declare a MINIMUM capability, so auto-update cannot break them silently. Compatibility is kept by rules plus a contract test that fails CI when a v1 field is removed, not by a promise. No code changes. The route, the promotion, the contract test, and the per-product migration follow as their own pull requests. --- docs/LOCAL-ENGINE-API.md | 209 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 docs/LOCAL-ENGINE-API.md diff --git a/docs/LOCAL-ENGINE-API.md b/docs/LOCAL-ENGINE-API.md new file mode 100644 index 0000000000..3e07f5b8d6 --- /dev/null +++ b/docs/LOCAL-ENGINE-API.md @@ -0,0 +1,209 @@ +# The local engine API + +Every Redrob product needs the same three things from this engine: a credential the +user only supplies once, model and provider routing, and — the reason this document +exists — the option of inference that does not leave the machine. None of those is +the agent. So the shared surface is a plain inference endpoint, and the agent stays +where it already is. + +This document is the design of record for that endpoint, for promoting the +`/experimental` surface, and for shipping the engine as one shared install instead +of a copy inside every product. + +## Why this is being written now + +Three products were already written against a local engine endpoint that has never +existed. + +- `redrob-browser` POSTs to `/experimental/external/chat/completions` and + streams the SSE reply (`sources/chrome/browser/redrob/sidecar_client.cc:592`). No + such route is served: the experimental group registers `capabilities`, `console`, + `tool`, `worktree`, `session` and `resource` and nothing else + (`packages/redrob/src/server/routes/instance/httpapi/groups/experimental.ts:91-101`). + The string `external` does not appear anywhere under `packages/redrob/src/server`. +- `redrob-reblend` ranks a LOCAL transport at `http://127.0.0.1:4096` and health-probes + it, but its sidecar rejects any non-HTTPS base URL, so that transport cannot carry a + request even when the probe succeeds. +- `redrob-office` compiles this engine from source at release time and ships it inside + its installer, with a packaging guard that refuses to build without it — and never + calls it. + +The products that DO drive the engine successfully (`redrob-cowork`, `redrob-cad`) +drive its own session API. The ones that tried to give the engine an OpenAI face are +the ones that are broken. That is the signal this design follows: serve the OpenAI +face for real, at one stable path, and stop pretending the session API is one. + +## 1. `POST /v1/chat/completions` + +The core path. There is no alias: `/experimental/external/chat/completions` is not +served, and `redrob-browser`'s call site moves to `/v1/chat/completions` in the same +wave (see Migration). + +### Where it sits + +On `packages/llm` and the provider registry — NOT on `Session`. A chat completion is +stateless and a session is not; mapping one onto the other is what forces an +ephemeral-session-per-request design and a fake OpenAI face over an event bus. The +route resolves the provider and credential exactly as the agent does, then makes the +model call directly. + +This is also what makes local inference work: a locally served model is just another +provider the registry resolves, and callers see no difference. + +### Request + +An OpenAI chat-completions subset: + +| field | required | notes | +| --- | --- | --- | +| `model` | yes | `redrob/auto` routes through the Console; a local provider id targets a local model | +| `messages` | yes | `system` / `user` / `assistant` / `tool` roles | +| `tools` | no | **presence changes who executes them — see below** | +| `tool_choice` | no | `auto` \| `none` \| a named function | +| `stream` | no | default `false`; `true` returns SSE | +| `max_tokens` | no | | +| `temperature` | no | ignored for models that fix their own sampling | +| `reasoning_effort` | no | passed through where the provider accepts it | + +Unknown fields are ignored rather than rejected, so a newer client can talk to an +older engine without a version check. + +### Tool ownership is decided by the request + +This is the whole design. One rule, no modes to configure: + +- **`tools` present → the CALLER owns them.** The engine does not execute anything. + It emits `tool_calls` and ends the turn with `finish_reason: "tool_calls"`. The + caller executes them and sends a follow-up request carrying `role: "tool"` + messages. Standard OpenAI semantics, and stateless. +- **`tools` absent → the ENGINE owns them.** Present behaviour: the engine selects + and runs its own tools and returns final text. + +In v1 the two do not mix. Declaring `tools` disables engine-side tools. One +predictable rule beats three configurable ones, and it is the rule every caller +already assumes. + +Why it has to be this way: `redrob-office`'s tools edit an open document held in the +renderer. `redrob-browser`'s tools drive a live tab. Neither can be executed by the +engine — handed over, the engine would edit the disk instead of the document. Caller +ownership is not a preference, it is the only correct answer for those hosts. + +### Streaming + +`stream: true` returns `text/event-stream`: OpenAI delta chunks, then +`data: [DONE]`. Tool calls stream as `delta.tool_calls` fragments keyed by index, the +way callers already parse them. + +### Auth + +Inbound uses the server's existing HTTP Basic (`packages/redrob/src/server/auth.ts:18`). +Callers do NOT send a Console key. + +Outbound uses the engine's OWN Console credential — `REDROB_API_KEY`, then the +credential store, then `auth.json` (`packages/core/src/console-key.ts:34-72`). This is +the single-login property: one `redrob providers login` and every product is +authenticated, because no product holds a key of its own on this path. + +### Errors + +The OpenAI error envelope, so existing client error handling works unchanged: + +```json +{ "error": { "message": "...", "type": "...", "code": "..." } } +``` + +One code is load-bearing: when the engine itself has no Console credential, the reply +is `401` with `code: "engine_not_authenticated"`. That is the only condition under +which a client should offer a sign-in action. Clients must branch on this code and not +on the message text, which is localized. A client that offers sign-in on any failure +reports a network timeout as "please log in" — that shipped, and it is what this code +exists to prevent. + +## 2. Promoting `/experimental` + +Every `/experimental/*` route is promoted out of the experimental namespace and +becomes part of the stable surface. + +This is safe to do as one breaking rename, because **no product calls any of them.** +The only callers are this repo's own generated SDKs +(`packages/sdk/js/src/gen`, `packages/sdk/js/src/v2/gen`), which `httpapi-codegen` +regenerates from the route definitions. The one external reference to an experimental +path is `redrob-browser`'s dead completions call, which this design replaces anyway. + +So: rename the paths, regenerate the SDKs, and delete the experimental namespace. No +compatibility window is owed to callers that do not exist. + +## 3. One shared install, no bundling, no pin + +Products stop shipping a copy of this binary. `redrob-office` drops ~145 MB from its +installer and drops its packaging guard; `redrob-cowork` drops its download-and-stage +step. + +### Resolution order + +1. `REDROB_CODE_BIN` — explicit override, development +2. `$HOME/.redrob/bin/redrob` — where `install.sh` already installs +3. `PATH` +4. absent → the product offers a one-click install that runs the official installer + (GitHub Releases, verified against `SHA256SUMS`) + +### One daemon, not one per product + +A shared record at `$HOME/.redrob/run/server.json`, mode `0600`: + +```json +{ "port": 51234, "pid": 8412, "secret": "…", "version": "0.1.7" } +``` + +The first product that needs the engine starts it; the rest attach. Required +behaviour: liveness check before attaching, recovery from a stale record whose pid is +gone, and reconnection by attached products when an upgrade restarts the daemon. + +Per-product processes were considered and rejected: with local inference, N processes +means the same weights loaded N times, which defeats the reason for doing this. + +That file is the auth boundary — it carries the Basic secret. It is per-user by +location and must stay `0600`; a multi-user machine gets one daemon per user, never a +shared one. + +### Capability negotiation replaces pinning + +No product pins an engine version. Auto-update would otherwise break clients +silently. Instead `/capabilities` (promoted, formerly `/experimental/capabilities`, +handler at `packages/redrob/src/server/routes/instance/httpapi/handlers/experimental.ts:39-41`) +grows an entry: + +```json +{ "chatCompletions": { "version": 1, "callerTools": true, "stream": true } } +``` + +A product declares the MINIMUM capability it needs, never a version number. If the +entry is missing or its `version` is below that minimum, the product tells the user to +run `redrob upgrade` instead of failing with a transport error. + +### Backward compatibility, as rules rather than a promise + +- The contract is versioned by PATH (`/v1/...`), never by engine version. +- Adding a request or response field is allowed. Removing one, or changing what one + means, requires `/v2`. +- When `/v2` appears, `/v1` keeps being served for two minor versions. +- A contract test in this repo pins the v1 request and response schema and fails CI + when a field is removed or its type changes. Without that test "we keep + compatibility" is a sentence, not a guarantee. + +## 4. Migration + +| product | change | +| --- | --- | +| redrob-browser | point `sidecar_client.cc:592` at `/v1/chat/completions`; its host-owned tool loop already matches the contract | +| redrob-office | drop the bundled binary and the packaging guard; point the engine route at the local daemon; keep its editor-owned tool loop | +| redrob-cowork | drop the download-and-stage step and the duplicate alias copy; keep driving the session API for agent work | +| redrob-reblend | either pass the local base URL and allow loopback HTTP in its sidecar, or delete the LOCAL transport that cannot work | +| redrob-canvas, redrob-query, redrob-recall | optional: switching from the Console to the local daemon buys them one shared login | +| redrob-extension, redrob-cad | unchanged; they want engine-owned tools and the session API already serves that | + +## Open items + +- Whether `tools` and engine-owned tools may ever be combined in one turn. Deferred + out of v1 deliberately. +- Which local inference providers the registry should resolve first.