crew: count a pinned checker in the panel's coverage warning - #1501
Open
santoshkumarradha wants to merge 924 commits into
Open
santoshkumarradha wants to merge 924 commits into
santoshkumarradha wants to merge 924 commits into
Conversation
Asked about work it had just handed off on the new road, the conversation's tasks tool answered that no task had run, because it read only the old task graph. It now lists a run's tasks from the run's own store, named the way the rail names them, a task by its number and a part by its place under it, each with its state and the first line of its result, and an id reads one of them. The shipped listing of earlier tasks is kept beside it. The manual no longer describes an ask box that no screen opens.
… open their pages (#1231)
…operties, with the verdict basis recorded (#1220) The worker declares its checks, and a holds verdict rests on a recorded, auditable, zero-exit run of every declared command. The worker records each command's own exit status, an absent status reads as unknown and earns nothing, and a record opens with a line that marks it as one that records exits, so a declared check that was never observed is refused, including when the record is missing or empty. A record written before exits were recorded still holds by reading and names the checks it did not observe. A does-not-hold verdict is ungated, a task with no declaration gets a reading verdict, and the verdict basis is persisted. One quote-aware reader of "is this one command" lives in internal/approval and is shared by the proposal door, the checker's runner gate and the store.
… callback ports (#1237)
…cadence, and the harness that measured it (#1235)
…ver the conversation (#1244)
…ntroducing itself again (#1236)
…her's keys (#1248) A profile write now takes a cross-process lock on a stable config.json.lock for the whole read, copy and rename, inside the in-process write mutex. The lock rides internal/filelock, so it holds on every platform the repo builds for. A write waits at most two seconds for another process and then fails with a plain timeout error. The lock file is never unlinked: a crashed holder is released by the operating system closing its file, so there is no stale lock to reclaim. Tests run two real processes forced to contend and show both keys survive, a bounded timeout, and recovery after a holder exits without unlocking. Those forced tests are unix-only; their clocks are sanity bounds, not load bounds.
…ump (#1249) Both tests dumped the whole environment through the capped output collector and read variables out of it. On a machine with a large ambient environment the dump is cut before the TMUX_TMPDIR line, so the test read it absent while the job shell had it, and the two checks for stripped variables passed without proving anything. The shell now prints the three variables on one short line with an explicit word for unset, through the same login shell road the product uses, and the test fails if that line is missing from the captured output. Test only; the product was already correct on every platform.
…llows on a beat (#1251) The run's summary is a read nobody pressed for, and its refresh can wait on a model for seconds. It was asked through the one ordered line that carries a person's gestures to the engine, so a press on a run's row, or a stop, waited behind it: 2.4 to 10.1 seconds on a real screen, 0.09 to 0.18 after. The summary now goes beside the line. What may leave the line is decided by property: the ask was not a gesture and nothing a person does next depends on the engine having seen it first. A law lists every door that stands in the line and fails when one is added without a reason. An open page on a task that can still move was re-read on every paint tick once the last read answered, 509 wire reads in ninety seconds for one page. It now follows on the rail's own beat, counted from the last read for any reason, and a page on an ended or held task is never read again. The follow stays in the line because its fold replaces the page and must not overtake a note.
A run under the task belt was started on a context nothing could cut, and no cancel was kept. Its rows wear task numbers, but a stop by number went only to the task graph, which has never held a run's rows. Measured on the real binary in hosted mode: the stop card answered that there was no such task, the run's own page answered with the store's sentence about who owns the root, ctrl+c and /quit closed the window, and the run carried on to its own landing every time. The id a surface already sends is now resolved to whoever owns the row, so an older window stops a run through a newer engine. A stop of a run ends the run's task and everything open under it in one store write, then cuts the run's context so workers and their calls end at once, then commits what was made on the run's own branch and gives the copy back. Nothing goes into the person's folder, no model turn is bought, and the person is told where the work is and how to bring it in or drop it. A hand-off that joined a run is stopped alone. A stopped run buys no further summary. On the surface: x on the run's own page raises the same stop card, x typed while that page is still loading reaches it too, the page no longer offers a hold the store would refuse, and ctrl+c is read above both the loading page and the open page. A law lists every place that publishes a running row and fails until each has a test that a stop on that row ends its owner. What ctrl+c, /quit and a closed window do to a live run is unchanged here.
FINDINGS.md at the repository root is a working note from the cell behind the unread config key notice. It is not product documentation and nothing reads it. A cell's findings belong in its run folder, outside the repository.
…h guard Only calls keyed to one of the crew seats walk the seat ladder unguarded; any other call the run makes goes through the route-health guard, so no call in a task reaches a quarantined or unaffordable route. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ction on a cut-off failure A free rescue takes a general or code model before one whose name marks it tuned for a single domain (finance, medicine, law and the like). The free notice is one clause. A failed task that saw an account out of credit or a refused key ends on that action rather than offering a stronger redo. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
WithoutPatientRateLimits takes patience off a call and hands a 429 back where the loop would otherwise wait on the same machine; a free move to another machine is still made at once. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…-move line Tiny reads a model size tag from its name; more domain tags are known. A seat ladder puts qualified rivals within 3x the pick cost ahead of dearer ones. A crew line says each seat net move with the last reason and how many were tried between, not every rung. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… pinned id The free rescue drops domain-tuned and tiny models. A model indexes are filled from any catalog row of the same model and from the evidence snapshot, so the undated row does not lose them. The crew migration retires only preset and pick words and auto rows; any model id in a seat row stays a pin, and the notice names the kept pins. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… errands guarded Crew seat calls hand a 429 back and move on; a seat tries at most three free pools in a task, so every route down stops on its action in seconds. Only the person own turn on their own model skips the route-health guard, so an errand whose floor is the conversation model is not sent to a quarantined route. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…r rung - Labels are read word by word with emoji shortcodes folded, so a tracker's `bug 🐛` is a bug label; a bug label with no open-ended label is a bugfix instead of the uncertain open-ended default. - Title signals: what the code does not do (never, does not, fails to, should ...) and a call written out in the title; "should support" and kin weigh back on the open-ended side. - A bugfix is read again for reach: more than one source file, an API or protocol, language rules, a long report or several repros, existing tests that must keep passing. Two or more make it complex; the worker then takes one rung up its front (nextRung), planner and checker unchanged, pins never overruled. Decision.Subclass/Reach and CrewRecord task_subclass/task_reach carry it to the log; lines still say bugfix. - The net-move line names the first cause: "(credit unavailable on openrouter; +3 tried)". - Open-ended checker cost priors move to p75 (kimi-k3 $0.0882). - Tests: kornia-4814 as filed and title-only read bugfix; hermes-7680 reads complex and a one-line typo simple; a complex fix gets a stronger worker at the same lambda; a pinned worker stays. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- CrewSpendCap: the crew cap, and the daily limit where the run is bound by it, whichever is lower, with the action a stopped call ends on. - CrewSeatCeilings: the checker's ceiling on one task, max(3x its estimate, $0.05); none when another seat shares the checker's model id, since the guard keeps spend per model. - A catalog row with no published price, or a non-:free id priced at zero (stealth/preview, meta-routers), is never a candidate or a free rescue; only an explicit-zero :free pool is free. Pins still name such ids. - CrewRouteAnswers for the provider route gate, with a fast path when nothing is unhealthy; a logged route failure drops the health cache so the next call sees it. - The migration notice lists kept pins worker, planner, checker. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- SpendGuard prices each seat call before it is made (prompt bytes/4 at the prompt or cache-read price, plus the completion ceiling) against the day's cap and the checker's own ceiling; a call that would cross either is not made and the crew stops on the one action. Actual cost is booked after. TUI crews use the crew cap; codeaf do adds the daily limit unless -yes-spend lifted it. - A send that failed its first call on this task is not asked again by any seat: the seat walks on, or ends on its action, without a request. The failure row is written once. - provider.Config.RouteGate is asked before every body the client sends; the session refuses a model whose route health says it will not answer, on every road, except a crew seat's own call and the person's turn. - Tests: checker stops at its ceiling and the day at its cap; every route down asks each free pool at most once across all seats and a retry; with a 403 home, no road reaches the refused route after the seat's own call. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The /crew panel, its notes and the crew line draw pin, tick, ring, cross and diamond from the plain tier whatever the icon setting says; a private-use codepoint is a blank cell without the font. - Manual: a call that would cross the cap is not made; the checker's per-task ceiling; complex fixes; label spellings; pools rested for the task; unpriced models never picked; the first cause on the net line. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- RouteCrew classified first (for the learned offset) and handed Decide only
the class, so Decide built Reading{Class, "given"} with no reach and every
complex fix ran on the simple fix's worker. Request.Reading now carries the
whole reading, RouteCrew passes it and the task, and a class given without
a reading still has its reach read off the task's words.
- An unmeasured model must publish at least one index the seat weighs to be
a pick or a rung; one with none (upstage/solar-mini4 on the planner) was
ranked on price alone. The last-rung rescue still takes it.
- Unmeasured q stays capped under the measured median (gpt-5.6-luna 3.973,
qwen3.8-27b 3.975 are their own published indexes, not backfill) and their
weighed cost is floored at the cheapest measured cost, so a measured model
wins at equal price; tested.
- Tests: Decide with Class given and with the reading; RouteCrew and
ResolveSeats (the codeaf do road) route the hermes-7680 report complex
with a stronger worker.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…p ends the turn Reproduced in the real TUI against a stub with a stack dump at the provider client: every crew guard bailed on ProfileDir == "", which is what every ordinary launch has (config.ProfileDir is the override variable; "" is the home profile). healthyModel, the route gate, the spend guard and CrewHealthySend/CrewRouteAnswers were all off, so the run summary (RefreshRunSummary, callRole worker, 320 widened to 6400 by the thinking share) asked the refused glm route, and the cap was never checked in a task. - The empty profile is the default profile on every crew path. - A crew's last word (crewStopped, ErrSpendStopped, errRouteResting) ends the turn at once. Weighed as the 429 underneath it, the turn sat out about a minute of backoff and asked the crew again: the all-down idle. - Helper calls in a conversation with crews are priced against the crew's day cap like seat calls, never wait out a rate limit, and are not made for a task that already stopped on its action. - The day holds every in-flight call's estimate and estimates a call at no less than the model's last actual cost today, so concurrent seats and a price sheet far under the real charge cannot pass the cap. - The task's crew line and log row carry helper spend: the helpers made for it, or the day's guarded spend since the task was routed when more. - Tests on the default profile through the real session: a 403 home asks the refused route only on the seat's first call (helper and a direct client refused); a $0.01 cap at $0.004 a call makes two calls and refuses the third and the line carries the helper; a crew stop wrapping a 429 with an hour's Retry-After ends the turn at once, asking once. A law that every provider client in the package is built through clientConfig, which sets the gate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Remove the experiment data under docs/design/model-pool/data and the scripts that produced the figures and tables (figures.py, paperstyle.py). The paper source, its figures, tables and the built pdf stay, and the Makefile now only builds the pdf. data/seed-cells.csv stays: the pool index seed generator reads it. The router prior (internal/crewroute/prior.json) is unchanged apart from its source notes, which now point at the paper. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comments in internal/crewroute (route.go, classify.go, prior.go, route_test.go) now state the rule the router follows, not the figures behind it: narrow fixes use the cheapest qualified crew, and open-ended work gets a strong checker. The knee comment says what the knee does (a step up is bought only above [Knee] points per dollar) and no longer lists per-crew deltas and costs. Four test failure messages in route_test.go now say what was expected instead of citing evidence. prior.json keeps all of its data; its source and costs_source notes now read "measured crews per class of work; see the Pareto Crewing paper". The spend guard's comment and its test lead-in describe an overrun without the incident's figures. The manual's crew estimate paragraph and the #1436 change entry refer to the router's table and the design paper instead of a trial. No behaviour or user-facing string changes; gofmt and go vet are clean. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The paper's LaTeX source, bibliography, generated tables (tab-*.tex), figure PDFs (fig-*.pdf) and the Makefile that built them are removed. docs/design/model-pool keeps pareto-crewing.pdf, RUNBOOK.md and data/seed-cells.csv, which internal/pool/index/cmd/seedgen reads. Nothing else in the tree referenced the removed sources; only the removed files referenced each other. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…5 limit Replace the shipped per-model table and hand-set scoring in internal/crewroute with a learned prior read from catalog metadata. crewroute: - prior.json now holds fitted weights only: a joint Gaussian over ability and catalog features (log prices, log context, release date, open weights, the three published indexes, arena Elo), family offsets, a per-(class, seat) linear link from ability to quality, per-seat token shapes and cost scales. No per-model rows ship. - quality() conditions on the fields a row has and returns a mean and a standard deviation; missing fields widen the variance. credible() needs a finite-variance score (evidence ratio) and an upper bound over the floor. - Cost comes from catalog prices x seat shape x class scale x the install factor. costFloor is the cheapest credible priced model; unpriced models are weighed at a priced model of equal ability. - Remove Snapshot/Measured/IsMeasured, seatWeight, unseenMargin, unseenShrink, the measured tie-break and the price ceiling. Every catalog model is a candidate. - Request.Learned applies per-install offsets; Request.TaskCap keeps the pick under the per-task limit (estimate x install factor), --best included. Note reads "held under the $X task limit". - Model gains ArenaElo and Released; ability memoised per model. router/config: - CrewRecord.Learned and CrewLog.Quality: accepted, kept, redo and failed outcomes move a per-(class, seat, model) offset by a bounded step (rate 0.1, bound 1 quality point), recorded on the decision row. - catalog reads `created`; crewModelOf sets Released from it or the slug date. Rows with indexes added are re-scored on the next refresh. - Per-task spend limit models.crew.task_cap (default $5): pre-call guard on every task call, stop line "this task reached its $5 limit · raise it in /crew", /crew cap row "per task $5 · daily $X", `/crew cap task <$>`, `codeaf do` holds it and -yes-spend does not lift it. tui3: Spending tab's per task row reads the /crew limit. Crew panel fixtures carry release dates and Elo; snapshots follow the new picks. Tests: replace the table-reproduction and effort-evidence tests with behaviour tests (stronger-indexed similar-priced model wins a seat, no metadata is not picked unless pinned, install evidence moves but does not freeze a score, bounded outcome learning, --best under the task limit, decision under 2 ms, weights under 2 MB). Manual and changelog updated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ach on one-paragraph asks crewroute ability: - A row that publishes any index (AA intelligence/coding/agentic, arena Elo) is conditioned on those indexes only; price, context, date, licence and family no longer move it. A row with none is conditioned on context, date and licence plus the family offset, and its mean is capped at the population mean with the excess added to the variance. Price is not an ability feature. prior.json v2 carries index_features/base_features. - Seats score the solve rate at theta - 1 sd (riskKappa); quality is no longer clamped to [0, 10], so ordering is preserved. - The checker needs mean u >= u_floor whenever any candidate reaches it (seatCredible); Gaps uses u + sd against the open-ended u_ref. - Link refitted with seat pooling within a class (support slopes shrunk toward the worker's), so no seat has a zero slope; --best buys the top model in every seat under the task cap. classify: - Reach signals read the title when a task has no body. - New reach signal: security fixes (traversal, injection, bypass, ...); "N repros" counts; keepPassing allows words between "existing" and "tests". - Title failure words add wrongly/rejected/instead of; a rename or version bump reads as a simple fix. Tests: dominance property test (no dearer and >= on every shared index never loses, 400 random pairs plus the v4.1-flash/glm-5.3-flash case), fix checker not the cheapest thin model, checker floor, --best strongest per seat, index-less row capped at the population mean, one-paragraph reach (T03/T11 texts), defect words and renames. Knee test now asserts open-ended buys a stronger checker at prices a fix does not. Config and tui3 expectations follow the new picks. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…and cheap tolerance - checkerFirst: on open-ended and other work at λ > 0, when the planner and checker slopes are within one SD of each other, the stronger of the two support picks sits the checker seat and the weaker the planner. Pins are never moved; a model that cannot sit the other seat stays. --best (λ = 0) is unchanged. - seatCredible: the worker, like the checker, needs mean u >= u_floor whenever any candidate reaches it (abilityFloor). Fallback ladders are exempt, so a seat that cannot start still moves to a similar cost. - --cheap: strongerWithin trades the worker pick for the strongest eligible model costing at most 1.5x (cheapTolerance). The knee comparison for the effort note runs without it. Tests: the checker is never weaker than the planner at the knee on open-ended and other work, and a pinned planner stays; a cheap worker is credible (glm-5.3-flash over a coding-only model) and the tolerance buys a stronger worker at 1.2x cost but not at 2.4x. Manual and changelog updated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
dev now carries santos/dev2 (#1410) and #1388, #1426, #1440 and others. The PR's net change is taken from its fork point bdb08cf; the earlier dev2 merge and its revert are left out, so dev's skills (#1396) and worker step-boundary work stay as dev has them. Conflicts: config/crew.go, config/seats.go and tui3/crew.go take the routed-crew versions; /crew keeps dev's notice event. do.go keeps both the kept-record line and the crew outcome log. task_run_belt.go keeps both the machine-hold rail and the crew completer. home.md keeps dev's rows without the preset forms and adds the crew-panel row. truth_test.go keeps dev's new facts and counts the crew as three seats. back_test.go stays deleted as on dev. CHANGELOG.md keeps dev's released history. #1440 on the routed crew: reflex and small work keep their free defaults on a balance read as low; the three crew seats see the OpenRouter account as out of credit before any call (not re-probed per task) and route to free pools with the free-routes notice. Tests follow. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With HOME and CODEAF_HOME empty, all three seats are routed, nothing is pinned, the allowed rule is all, a task is held to $5 and there is no daily crew cap. With nothing routable, no crew seat falls back to a built-in model and resolving a crew is an error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Names the four kinds of work (bugfix, complex fix, open-ended, other), the candidates (every model a connected provider serves) and learned weights; draws the panel's cap row as "per task $5 · daily none" and says the seats' "usually" reads the last eight tasks; adds a route health section, the /crew cap task row, the per-task limit on codeaf do (-yes-spend does not lift it), and the crew's limits in LIMITS.md. openrouter-credits.md says how a low balance reaches the routed crew. The changelog entry names the classes, the #1440 change and the redo decay as the code has it. Onboarding comments no longer mention a crew step. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The two-millisecond budget was checked against one batch's mean, which on a loaded shared machine also counts scheduler preemption. It now takes the fastest of five batches of forty. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
crewroute.Gaps asked only whether any allowed, seatable model met the strong-checker test. A pinned checker always runs, so the answer ignored the seat that actually checks: a strong pin over a weak set (or a pin whose route is quarantined and so missing from the candidates) kept the "no strong checker among the models you allow" warning, and a weak pin over a strong set showed none. Gaps now takes the pinned seats. With the checker pinned, the pin alone is tested (read from the candidates by lineage when they carry it, else from the catalog figures config passes): strong gives no gap, weak gives "checker pinned to <model> · open-ended work will be checked weakly". An unpinned checker is judged over the allowed models as before. The checker is the only gap kind today, so there is nothing else to adjust. config.CrewGapsAt reads the pins and resolves each through the catalog. The manual names both warning lines. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
santoshkumarradha
marked this pull request as ready for review
September 25, 2026 14:59
Member
Author
|
@AbirAbbas small follow-up to #1436, ready for review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong.
crewroute.Gapsasked only whether any allowed, seatable model passed the strong-checker test. A pinned checker always runs, but the check never looked at it. Pinning a strong checker over a weak set, or pinning one whose route is quarantined and so absent from the candidates, leftno strong checker among the models you allow · open-ended work will be checked weaklyon the/crewpanel. Pinning a weak checker over a strong set showed no warning at all.Fix.
Gaps(candidates, pins map[Seat]Model): when the checker is pinned, only the pin is tested. It is read from the candidates by lineage when they carry it, the waypinnedreads it, and otherwise from the figures passed in. A strong pin gives no gap. A weak pin giveschecker pinned to <model> · open-ended work will be checked weakly. An unpinned checker is judged the same way as before. The checker is the only gap kind today, so no other kind needed the same change.config.CrewGapsAtreads the pins and resolves each one through the catalog, so the result does not depend on the pin's route health.models-and-cost.md) now names both warning lines. The changelog entry is1501-crew-pinned-checker-gap.md.Tests
crewroute/TestGapsCountAPinnedCheckercovers four cases: a strong pin over a weak set gives no gap, whether the candidates carry the pin or not; a weak pin over a strong set gives a gap that names the pin; with only other seats pinned, the unpinned checker behaves as before.config/TestTheGapCountsAPinnedCheckerworks end to end on a profile: a strong pin whose route is quarantined gives no gap, a weak pin gives the named line, and unpinning brings back today's answer.tui3/TestCrewSnapshotWeakPinnedCheckeris a panel snapshot with the named warning under the rows. The existingTestCrewSnapshotPanelalready covers a strong pin with no warning.All of these were run on
b1f125a, withOPENROUTER_API_KEYunset (#1489), and all passed:make vet fmt-checkgo testforcrewroute,config,tui3,manualandcmd/codeafmake manual-gates test-laws changelog-check test-packed-manualThe diff touches only those packages, so the full
make checkwas not needed.Stacked on #1436; retarget to dev once #1436 merges.
🤖 Generated with Claude Code