Reach the console's multi-model endpoints, and print what they cost - #43
Merged
Merged
Conversation
…cost
Two commands, `variants.paraphrase` and `variants.compare`, plus the route,
service and schema they need. Both are gated on the Redrob provider being
connected: the route answers 404 when it is not, so a user on a local runtime or
another vendor is told the feature is unavailable rather than watching a request
fail for reasons that look like their own.
THE COST IS ALWAYS SHOWN. One of these requests makes several charges, one per
model. A user who believes they made one request will be surprised by the bill
unless the surprise happens immediately, while they can still decide not to do it
again. So every result prints the model that actually answered -- `routedModel`,
which under `auto` is the only thing that names it -- with its own cost, its
latency, and the total.
`compare` offers the answers in a selection list with previews, and only the
pick enters the conversation. Session history cannot be pruned: `Revert.State`
rewinds to a point and restores FILES with it, and `fork` makes a new session,
so neither can remove "these two of three messages". An answer appended to try
it out would be permanent.
Driven by hand in a terminal, which is the only thing that found three of these:
- The result was a toast. The toast is absolutely positioned, capped at sixty
columns and dismisses on a timer, so it clipped the cost total and the whole
answer and kept only the first few slot rows -- the two things a reader most
needs were the two it dropped. Now a dialog, dismissed by the person reading
it.
- `compare`'s chooser never opened. `DialogAlert` calls `onConfirm` and then
clears the dialog stack itself, so a chooser pushed synchronously from that
callback is opened and immediately wiped. Deferred by a tick.
- Picking an answer appeared to do nothing. The insert was a `void`ed promise
with no catch, so its rejection was discarded. A user who has just been
charged for several models must be told when the thing they paid for failed
to land.
`CONSOLE_URL` is now resolved once, in the constants module, rather than at each
call site. A per-consumer `Flag.REDROB_CONSOLE_URL ?? CONSOLE_URL` made the
override HALF apply: the variants service honoured it while the session's own
inference path, the catalog fetch and the provider registration still went to
production. Pointing the CLI at a local console then produced one that answered
`/variants/paraphrase` locally and 401'd every chat turn against the real one --
a split that reads as a credential bug and is not. Measured after the change:
the session's own turn reaches the local console.
The HTTP client is provided inside the variants service instead of being
demanded from callers. The requirement was propagating out through the route
handler into the API's own requirement type and out to every entry point that
builds the API; `serve` failed to typecheck with `Type 'HttpClient' is not
assignable to type 'Service'`, which reads as a problem with `serve` and is not
one. The repository's usual shape is a service node with
`deps: () => [..., httpClient]`, sharing one client process-wide; these are two
plain functions, so they take their own. That is a real difference and is
recorded at the call site.
`console-key.ts` extracts the credential resolution both consumers need. There
are three origins and nothing bridges them -- `REDROB_API_KEY`, the Integration
store, `auth.json` -- with different shapes (`type:"key"` versus `type:"api"`).
Reading only some of them already shipped once as a bug: desktop-app users saw
six fallback models.
Also `loop.start` / `loop.stop`: a goal-driven autopilot. The first version
re-sent a fixed instruction, which cannot notice circling because the input
never changes. Each cycle now sends the goal, the ledger of approaches already
rejected, and an instruction to assess against the goal before taking the next
step. It stops on goal met, blocked, a repeated plan (three times, since two is
patience -- waiting on a build looks identical), a cycle backstop, or a session
failure. Driven by `session.idle` rather than a timer, because a timer fires
into a session that is still working and stacks turns.
The HttpApi exerciser gate fails on any route with no scenario, and it caught both of mine. Exercised through their NOT-CONNECTED path, which is the honest state in CI: there is no console credential, so the assertion is the 404 saying the Redrob provider is not connected. A scenario that needed a real key would either skip -- and this gate fails on skip, correctly, since a skipped route is an unexercised route -- or send a paid request to the live console on every run. 404 rather than 401 is the contract being pinned: nothing is configured to ask, so nothing refused us.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires the console's multi-model endpoints into the CLI as two commands, and adds a goal-driven autopilot loop.
Needs console #127 deployed before
paraphraseworks against production — that PR fixes three bugs this client is what found.variants.paraphrase/variants.compareBoth gated on the Redrob provider being connected: the route answers 404 when it is not, so a user on a local runtime or another vendor is told the feature is unavailable rather than watching a request fail for reasons that look like their own.
The cost is always shown. One request makes several charges, one per model. A user who believes they made one request will be surprised by the bill unless the surprise happens immediately, while they can still decide not to do it again.
routedModelis what names the model that actually answered — underautoit is the only thing that does.compareoffers the answers in a selection list with previews, and only the pick enters the conversation. Session history cannot be pruned:Revert.Staterewinds to a point and restores files with it, andforkmakes a new session — neither can remove "these two of three messages". An answer appended just to try it would be permanent.Three bugs that only a real terminal found
compare's chooser never openedDialogAlertcallsonConfirmand then clears the dialog stack itself, so a chooser pushed synchronously from that callback is opened and immediately wiped. Deferred by a tick.voided promise with nocatch; the rejection was discarded. A user who has just been charged for several models must be told when the thing they paid for failed to land.Verified the insert lands by reading the session store rather than the screen:
REDROB_CONSOLE_URLnow applies to the whole CLIA per-consumer
Flag.REDROB_CONSOLE_URL ?? CONSOLE_URLmade the override half apply: the variants service honoured it while the session's own inference path, the catalog fetch and the provider registration still went to production. Pointing the CLI at a local console produced one that answered/variants/paraphraselocally and 401'd every chat turn against the real one — which reads as a credential bug and is not.Resolved once in the constants module. Measured after the change: local console inference calls went
0 → 2on a session turn that previously 401'd.An HttpClient requirement that reached
serveThe
HttpClientrequirement propagated out of the variants functions, through the route handler, into the API's own requirement type, and out to every entry point building the API:That reads as a problem with
serveand is not one. Provided inside the service instead. Correcting myself: I earlier reported these two typecheck errors as pre-existing on develop. They were not — clean develop typechecks at 0, and both were mine.console-key.tsExtracts credential resolution both consumers need. Three origins, nothing bridges them —
REDROB_API_KEY, the Integration store,auth.json— with different shapes (type:"key"vstype:"api"). Reading only some already shipped once as a bug: desktop-app users saw six fallback models.Runtime finding that typechecking could not give:
Credential.Serviceis not in scope in HTTP handlers despite compiling. Resolved viaIntegration.Service.loop.start/loop.stopThe first version re-sent a fixed instruction, which cannot notice circling because the input never changes. Each cycle now sends the goal, the ledger of approaches already rejected, and an instruction to assess against the goal before taking the next step.
Stops on: goal met, blocked, a repeated plan (three times — two is patience, since waiting on a build looks identical), a cycle backstop, or a session failure. Driven by
session.idlerather than a timer, because a timer fires into a session that is still working and stacks turns.Not yet verified: whether models emit the
AUTOPILOT NEXT:marker reliably. If they do not, stall detection never fires. That is the most uncertain part of this design and wants a real run.Gates
Those three are unrelated to this branch and confirmed pre-existing by stashing everything and re-running.