Skip to content

Reach the console's multi-model endpoints, and print what they cost - #43

Merged
its-janghoon merged 2 commits into
developfrom
feature/console-variants-and-autopilot
Sep 21, 2026
Merged

its-janghoon merged 2 commits into
developfrom
feature/console-variants-and-autopilot

Conversation

@its-janghoon

Copy link
Copy Markdown
Contributor

Wires the console's multi-model endpoints into the CLI as two commands, and adds a goal-driven autopilot loop.

Needs console #127 deployed before paraphrase works against production — that PR fixes three bugs this client is what found.

variants.paraphrase / variants.compare

Both gated on the Redrob provider being connected: the route answers 404 when it is not, so a user on a local runtime or another vendor is told the feature is unavailable rather than watching a request fail for reasons that look like their own.

The cost is always shown. One request makes several charges, one per model. A user who believes they made one request will be surprised by the bill unless the surprise happens immediately, while they can still decide not to do it again.

Rewritten
  gpt-5.6-sol      $0.0005    4ms
  gpt-5.6-terra    $0.0006    9ms
  claude-opus-5    $0.0013   13ms
  total            $0.0024
  <the answer>

routedModel is what names the model that actually answered — under auto it is the only thing that does.

compare offers the answers in a selection list with previews, and only the pick enters the conversation. Session history cannot be pruned: Revert.State rewinds to a point and restores files with it, and fork makes a new session — neither can remove "these two of three messages". An answer appended just to try it would be permanent.

Three bugs that only a real terminal found

Result was invisible It was a toast — absolutely positioned, capped at 60 columns, dismissed on a timer. It clipped the cost total and the entire answer, keeping only the first slot rows. The two things a reader most needs were the two it dropped. Now a dialog.
compare's chooser never opened DialogAlert calls onConfirm and then clears the dialog stack itself, so a chooser pushed synchronously from that callback is opened and immediately wiped. Deferred by a tick.
Picking did nothing The insert was a voided promise with no catch; the rejection was discarded. A user who has just been charged for several models must be told when the thing they paid for failed to land.

Verified the insert lands by reading the session store rather than the screen:

session_input: "로봇의 정의는\n\n(answered by gpt-5.6-sol)\n\nHere is a short answer…"

REDROB_CONSOLE_URL now applies to the whole CLI

A per-consumer Flag.REDROB_CONSOLE_URL ?? CONSOLE_URL made the override half apply: the variants service honoured it while the session's own inference path, the catalog fetch and the provider registration still went to production. Pointing the CLI at a local console produced one that answered /variants/paraphrase locally and 401'd every chat turn against the real one — which reads as a credential bug and is not.

Resolved once in the constants module. Measured after the change: local console inference calls went 0 → 2 on a session turn that previously 401'd.

An HttpClient requirement that reached serve

The HttpClient requirement propagated out of the variants functions, through the route handler, into the API's own requirement type, and out to every entry point building the API:

Type 'Effect<void, unknown, HttpClient | Service>' is not assignable to 'Effect<void, any, Service>'
  Type 'HttpClient' is not assignable to type 'Service'

That reads as a problem with serve and is not one. Provided inside the service instead. Correcting myself: I earlier reported these two typecheck errors as pre-existing on develop. They were not — clean develop typechecks at 0, and both were mine.

console-key.ts

Extracts credential resolution both consumers need. Three origins, nothing bridges them — REDROB_API_KEY, the Integration store, auth.json — with different shapes (type:"key" vs type:"api"). Reading only some already shipped once as a bug: desktop-app users saw six fallback models.

Runtime finding that typechecking could not give: Credential.Service is not in scope in HTTP handlers despite compiling. Resolved via Integration.Service.

loop.start / loop.stop

The first version re-sent a fixed instruction, which cannot notice circling because the input never changes. Each cycle now sends the goal, the ledger of approaches already rejected, and an instruction to assess against the goal before taking the next step.

Stops on: goal met, blocked, a repeated plan (three times — two is patience, since waiting on a build looks identical), a cycle backstop, or a session failure. Driven by session.idle rather than a timer, because a timer fires into a session that is still working and stacks turns.

Not yet verified: whether models emit the AUTOPILOT NEXT: marker reliably. If they do not, stall detection never fires. That is the most uncertain part of this design and wants a real run.

Gates

typecheck   0 errors across the monorepo
packages/tui    281 pass / 1 skip / 0 fail
packages/core   970 pass / 3 fail  — the same 3 fail on clean develop
                (ModelsDev static fallback: limit.output expects 32000, gets 64000)

Those three are unrelated to this branch and confirmed pre-existing by stashing everything and re-running.

…cost

Two commands, `variants.paraphrase` and `variants.compare`, plus the route,
service and schema they need. Both are gated on the Redrob provider being
connected: the route answers 404 when it is not, so a user on a local runtime or
another vendor is told the feature is unavailable rather than watching a request
fail for reasons that look like their own.

THE COST IS ALWAYS SHOWN. One of these requests makes several charges, one per
model. A user who believes they made one request will be surprised by the bill
unless the surprise happens immediately, while they can still decide not to do it
again. So every result prints the model that actually answered -- `routedModel`,
which under `auto` is the only thing that names it -- with its own cost, its
latency, and the total.

`compare` offers the answers in a selection list with previews, and only the
pick enters the conversation. Session history cannot be pruned: `Revert.State`
rewinds to a point and restores FILES with it, and `fork` makes a new session,
so neither can remove "these two of three messages". An answer appended to try
it out would be permanent.

Driven by hand in a terminal, which is the only thing that found three of these:

  - The result was a toast. The toast is absolutely positioned, capped at sixty
    columns and dismisses on a timer, so it clipped the cost total and the whole
    answer and kept only the first few slot rows -- the two things a reader most
    needs were the two it dropped. Now a dialog, dismissed by the person reading
    it.
  - `compare`'s chooser never opened. `DialogAlert` calls `onConfirm` and then
    clears the dialog stack itself, so a chooser pushed synchronously from that
    callback is opened and immediately wiped. Deferred by a tick.
  - Picking an answer appeared to do nothing. The insert was a `void`ed promise
    with no catch, so its rejection was discarded. A user who has just been
    charged for several models must be told when the thing they paid for failed
    to land.

`CONSOLE_URL` is now resolved once, in the constants module, rather than at each
call site. A per-consumer `Flag.REDROB_CONSOLE_URL ?? CONSOLE_URL` made the
override HALF apply: the variants service honoured it while the session's own
inference path, the catalog fetch and the provider registration still went to
production. Pointing the CLI at a local console then produced one that answered
`/variants/paraphrase` locally and 401'd every chat turn against the real one --
a split that reads as a credential bug and is not. Measured after the change:
the session's own turn reaches the local console.

The HTTP client is provided inside the variants service instead of being
demanded from callers. The requirement was propagating out through the route
handler into the API's own requirement type and out to every entry point that
builds the API; `serve` failed to typecheck with `Type 'HttpClient' is not
assignable to type 'Service'`, which reads as a problem with `serve` and is not
one. The repository's usual shape is a service node with
`deps: () => [..., httpClient]`, sharing one client process-wide; these are two
plain functions, so they take their own. That is a real difference and is
recorded at the call site.

`console-key.ts` extracts the credential resolution both consumers need. There
are three origins and nothing bridges them -- `REDROB_API_KEY`, the Integration
store, `auth.json` -- with different shapes (`type:"key"` versus `type:"api"`).
Reading only some of them already shipped once as a bug: desktop-app users saw
six fallback models.

Also `loop.start` / `loop.stop`: a goal-driven autopilot. The first version
re-sent a fixed instruction, which cannot notice circling because the input
never changes. Each cycle now sends the goal, the ledger of approaches already
rejected, and an instruction to assess against the goal before taking the next
step. It stops on goal met, blocked, a repeated plan (three times, since two is
patience -- waiting on a build looks identical), a cycle backstop, or a session
failure. Driven by `session.idle` rather than a timer, because a timer fires
into a session that is still working and stacks turns.
The HttpApi exerciser gate fails on any route with no scenario, and it caught
both of mine. Exercised through their NOT-CONNECTED path, which is the honest
state in CI: there is no console credential, so the assertion is the 404 saying
the Redrob provider is not connected.

A scenario that needed a real key would either skip -- and this gate fails on
skip, correctly, since a skipped route is an unexercised route -- or send a paid
request to the live console on every run.

404 rather than 401 is the contract being pinned: nothing is configured to ask,
so nothing refused us.
@its-janghoon
its-janghoon merged commit 9346cdb into develop Sep 21, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant