feat(fusion): two slots, either kind — pick who orchestrates and who executes - #403
Open
plombeer31 wants to merge 9 commits into
Open
feat(fusion): two slots, either kind — pick who orchestrates and who executes#403plombeer31 wants to merge 9 commits into
plombeer31 wants to merge 9 commits into
Conversation
…work Fusion's split has been advice since the mode shipped, and a capable cloud model handed a catalog of forty tools reads the twelve files itself and never fans anything out. A QA run showed exactly that: nine steps, twelve os.fs.read calls, zero delegations, every completion on the cloud model. So the first mutation of a fusion turn is now refused until the turn has delegated once. Read, plan, split, delegate — then the gate opens, because two things genuinely belong to the orchestrator afterwards: integration, which is a write, and anything a worker handed up because it needed approval, which workers cannot request. A refusal rather than a hidden descriptor: removing tools mid-turn rewrites the stable prefix and drops the session's KV cache, while a refusal costs one tool result and reads as an instruction. Same trade plan mode makes, and the same one fusion.delegate already makes for a worker that calls it. Read-only tools, the fan-out itself and the terminal verbs are never gated — planning is reading, and vetoing reply would veto the turn's exit. The guidance is rewritten around the loop it now enforces: plan in the open with the parts sized, one task per part in one call, briefs that stand alone, read every reply against its brief, merge yourself, and send rework back out instead of quietly absorbing it. Same prefix budget as before (1344 of 1400 bytes). A worker's own turn is never gated: the flag is per turn and skips ephemeral ones, so the hands the mode exists to free stay free.
A first live run on a cheap auto-routed model read the files, replied 'no edits were needed', and changed nothing. Stopping after the plan is the cheapest way to satisfy 'plan in the open', so the line now says the delegation belongs to the same turn.
The loop built the per-turn fusion context and handed it to the step executor, which builds the batch context and forwards `isPlanMode` — and nothing else. The gate was live, unit-tested and never consulted: a driven run in fusion mode wrote all four files itself with zero refusals and zero workers. Forwarded the three fields the same way `isPlanMode` travels, together so a context can never carry the flag without the ledger behind it.
… fan-out Nobody should be picking the worker count from a list. It is two decisions and neither belongs to the operator: how many workers this machine can serve at once, which the machine knows, and how many a given job is worth splitting into, which the orchestrator knows once it has read the job. The machine half: `localModels.managed.parallel` takes `"auto"` and defaults to it (config v63). llama.cpp divides `--ctx-size` between `--parallel` slots, and a slot smaller than a worker's own prompt cannot serve one — so the count is how many times MIN_SLOT_CONTEXT (16k, the same figure as MIN_AUTO_CONTEXT) fits in the context the daemon is launched with, capped at 8, and 1 on a CPU-only launch where concurrent slots share the same cores. That context is already sized from VRAM by context-size.ts, so a bigger machine widens the pool with no new probe and nothing to configure. Resolved in daemon-lifecycle, where the effective context is known; both launch sites pass the config value through untouched. Migration: a pre-v63 file's `parallel: 2` is the schema's old default, not a choice anyone made, so it reads as `"auto"`. Any other number is someone's deliberate pin and is kept — as is a 2 written at v63 or later. The model half: a `fusion.delegate` call that names no `maxWorkers` now gets the machine's capacity instead of `llm.runMode.fusion.workers`. The pool notice was reworded to match — it names the slot count as where the number comes from rather than as a knob to go and raise. The composer's count rows are gone; its label reads "up to N workers", because N is the ceiling and not a promise about this turn. The worker MODEL rows stay: which model runs the workers is a real choice. `/runmode workers N` survives as the pin.
…executes Fusion assumed its pairing: cloud orchestrator, local workers. That is the right default and the economics the mode was built for, but it is not the only sensible pairing. Cheap local planning driving capable cloud executors is a real use case, and so is a big local model orchestrating a small one. The kinds were baked into three places; none of them had to be. - The resolver's worker default still prefers a local provider, but only as a default — a pinned leg is honoured whatever its kind, and the fallback is now "any provider that is not the other leg" rather than "a llama-server or nothing". - setMode no longer forces the orchestrator onto a cloud provider. An explicit pin wins; without one the order is the active provider, then the first usable cloud one. It picks the second leg the same way and refuses only when there is no second provider to pick. - The pre-flight counts usable legs instead of requiring one of each: a keyed cloud row can answer, a local row with something on disk can answer, and fusion needs two of those. - `no-local-provider` becomes `no-second-provider`, and its sentence stops prescribing a kind. In the composer the two controls are now the two slots. The provider control is the orchestrator: cloud rows as before, plus `local-llama` when a model is on disk. The workers control is the second slot: the models on disk, plus every cloud provider that is not already orchestrating — fanning out to the model doing the orchestrating buys nothing and doubles the bill. Picking a local model for the workers also claims the slot for the local leg, so the pick cannot be quietly ignored by a pin left on a cloud provider. The `### fusion` machine line only states slot counts when the local leg is actually running the workers; with cloud workers there is no slot pool to describe, and the idle daemon's number would be about the wrong machine.
The schema had the fourth copy of fusion's assumed pairing, and the worst one: `orchestratorProvider` had to be cloud and `workerProvider` had to be llama-server, enforced by throwing during config load. With the composer now able to write the swap, the next launch died before the TUI existed — ConfigValidationError: invalid config: llm.runMode.fusion.orchestratorProvider: orchestrator must be a cloud provider, "local-llama" is llama-server — edit or delete that file — which is an operator hand-editing JSON to get their agent back. Found by driving the shipped binary: pick the swap, quit, relaunch. The kind check is gone from both legs. What the schema still enforces is what it is good at: the id names a provider that exists. The one pairing that is genuinely wrong — both legs on the same provider — belongs to `resolveRunMode`, which sees both at once and degrades instead of refusing to start.
plombeer31
force-pushed
the
feat/fusion-any-leg
branch
from
September 10, 2026 16:41
ff74785 to
32b2a65
Compare
…ocal one Driving the swap on a built binary: pick `local-llama` for the orchestrator and the strip read `local-llama ⇄ qwen-3.5-4b · up to 2 workers` while the config said the workers were openrouter. Both halves of the right-hand side were describing the local daemon regardless of which leg the workers were on — the same class of lie as the chip that said fusion while the runtime was cloud. Three places, one cause each: - `resolveRunMode` filled `workerModel` from `managedModelId` unconditionally. That is the model the LOCAL daemon serves, so it now only answers for a local worker leg; a cloud leg reports its own default chat model. - The strip's worker half read the local models panel directly. It now labels from the worker leg's own provider row, the way the orchestrator half already did. - `up to N workers` is llama-server's request-slot count, which a cloud leg does not have. It reads `cloud workers` there instead of quoting a number about an idle daemon.
It read `up to 2 workers`, and on a narrow terminal it collided with the steer hint — `up to 2 work⏎ steer`. The columns bought nothing: the number is the machine's capacity, not a choice anyone makes, and how many workers a turn actually spends is the orchestrator's call on that turn. Both legs are already named by the model segment (`A ⇄ B`). The worker slot itself stays: it is still one ←/→ away inside the composer popup, which is where the ←/→ walk has always included it. Removed the whole prop chain rather than passing null through it — prompt-shell, prompt-meta-bar, ComposerMetaControls and the selector.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #398 (which stacks on #391); the diff to review here is the last commit.
Fusion assumed its pairing — cloud orchestrator, local workers. That is the right default and the economics the mode was built for, but it was baked in as a rule, and it did not have to be. Cheap local planning driving capable cloud executors is a real use case; so is a large local model orchestrating a small one.
So fusion now has two slots and either may hold either kind.
What was actually enforcing the pairing
Three places, none of which needed to:
setMode. It forced the orchestrator onto a cloud provider and refused withno-cloud-providerwhen there wasn't one. An explicit pin now wins; without one the order is the active provider, then the first usable cloud one. The second leg is picked the same way, and the refusal only fires when there is no second provider at all.no-local-providerbecameno-second-provider, and its sentence stopped prescribing a kind.In the composer, the two controls are the two slots
local-llamawhen a model is on disk.One honesty fix that came with it
The
### fusionmachine line states llama-server slot counts. Those are a fact about the local daemon, so it now states them only when the local leg is the one running the workers; with cloud workers there is no slot pool to describe and the idle daemon's number would be about the wrong machine.Verified
npx tsc --noEmitclean. New resolver tests for the swapped pairing (local orchestrator + cloud workers resolves to effective fusion with both legs pinned) and for two cloud providers never landing on the same leg. Every test that encoded the old assumption was rewritten to the new contract rather than deleted — the pre-flight's two blockers,setMode's two refusals, the fusion row's detail column, and the worker rows.src/tui,src/llm,src/prompt,src/tools/fusion,src/config: 4756 passing.