Let the spawn system read model modalities to understand the tasks it's good/bad at

[grzegorznowak]

Let the spawn system read model modalities to understand the tasks it's good/bad at.

• On the model-group level we should pull down the registered model capabilities (they don't change often per model so probably one-time download would suffice - need to confirm)
• A group of models should express the common set of modalities across all the models added, warn on empty set
• User should be able to alter the calculated modalities (but never add ones that do not exist)
• The main session need to be aware of the modalities per group for better routing
• The spawn should know about what modalities the model is good at to not start hacking around with vision via 3rd party plugins rather than fail early

NOTE:
an extension of this ticket might include aspects such as

Maybe also set it by the user at the group level so it can prevent you from adding wrong models?
Similarly I'd also want other protections for min context size, price, param count class, etc
So groups make engineering sense not just a hunch

Meaning we need to construct a solution that will be pluggable

---

Tech context — grounded capability/modality state (2026-08-20)

Verified against agenticoding/pi-agenticoding main HEAD 4efc9cf (Merge PR #20; PR #21 searchable/capped picker merged via d4a90f), dev floor @earendil-works/pi-ai / pi-coding-agent / pi-tui pinned 0.82.0 in package.json.

1. Per-model capability data already exists at runtime — it is just not surfaced.

• ModelGroupModel = { provider, modelId, thinkingLevel? } — model-groups/types.ts. Stored group definitions carry no modality/capability fields.
• pi-ai's Model<TApi> already exposes capabilities on every resolved model: reasoning: boolean, input: ("text" | "image")[], plus thinkingLevelMap, contextWindow, maxTokens, cost, compat — node_modules/@earendil-works/pi-ai/dist/types.d.ts (Model interface).
• ModelRegistry (pi-coding-agent dist/core/model-registry.d.ts) is a synchronous facade: find(provider, modelId) → Model | undefined, getAll(), getAvailable(), hasConfiguredAuth(model), async refresh() (reloads models.json). find returns undefined for unknown/unconfigured refs.

2. The same live lookup is already in use — no new plumbing needed.

• model-groups/router.ts — resolveSpawnModelRoute resolves each entry via modelRegistry.find(...) + hasConfiguredAuth(...) before selecting the routed model.
• model-groups/store.ts — validateModelGroups computes per-group unavailableRefs via the same lookup (boot snapshot surfaced in TUI + notify).
• model-groups/tui.ts — modelAvailable() (l.58), EDITOR model rows (l.510-515), MODEL_EDIT detail (l.521-535), LIST rows (l.485-497).
• The plugin already consumes model.reasoning in one place today: tui.ts:279 gates the off thinking option on it. Model.input is not read anywhere in plugin src yet — only in test mocks (tests/unit/helpers.ts).

3. Group-level modality set is derivable at read time; nothing persists it today.

• Common set = intersection of member models' input arrays + reasoning aggregation; trivially computed from the registry on each read. No schema bump / migration; stays fresh after registry.refresh() (matters for dynamic catalogs like OpenRouter aggregate providers). Persisting at def time would go stale — not recommended.
• ⚠️ Save-path caveat for any future persisted override: store.ts validateModelEntry drops unknown keys on write, so an older plugin version saving a config would silently strip a new field (schema-version guard needed before persisting overrides).

4. Where spawn / main session would consume it.

• Spawn routing already returns the concrete Model<Api> (router.ts SpawnModelRoute.model) — input/reasoning ride along for free. Fail-early modality checks hook into the existing SpawnRouteError boundary in router.ts or pre-session validation on the spawn tool path; they do not change spawn's current signature.
• Main session: AgenticodingState already holds modelGroups.groups + validation (state.ts); a derived per-group modalities map fits there without persistence.

5. Re "one-time download would suffice" from the ticket.

• The plugin does not need to download anything: modalities live on each Model entry in pi's models.json catalog and are exposed by ModelRegistry. Freshness = registry refresh, not a plugin-side fetch.

6. Pluggable extensions (min context size, price, param count class).

• The same read-time derivation extends to contextWindow, maxTokens, cost — all already present on Model — so a pluggable constraint/validation layer can consume derived per-group facts without schema changes.

Test / robustness anchors: tests/unit/model-groups-crud.test.ts (mock registry carries reasoning/thinkingLevelMap), tests/unit/model-groups-integration.test.ts (component rendered via .render(80).join + regex asserts; /model-groups is TUI-only, RPC/JSON/print rejected), tests/unit/model-groups-router.test.ts. TUI rule: no console output (ANSI corruption) — all dynamic text via theme.fg + truncateToWidth + escapeDisplayLabel (model-groups/display.ts).

---

Locked decisions (2026-08-20, issue session)

Decisions below were made with the operator and are authoritative for the implementation step. They are recorded here so the issue body is the single durable contract.

Q1 — Override persistence: persist (operator: "Q1 . save")

The per-group modality set the user edits is persisted in the stored model group config, not session-derived.

• Persisting requires a schema-version guard: config CURRENT_VERSION in model-groups/store.ts is currently 1, and validateModelEntry (store.ts:47) builds entries from exactly { provider, modelId, thinkingLevel? } while silently dropping any unknown key on save. A persisted modality field must be preserved through that path, or an older plugin version saving a newer config would strip it.
• Constraint (per issue bullet 3): the user may alter the calculated set, but never add a modality no member model actually supports — the override is capped at the union of member capabilities.
• Staleness: persisted overrides must be reconciled after a catalog refresh (derived facts can change); the effective set stays live-derived where applicable.

Q2 — Spawn fail-early: declared requiredModalities, reusing the mechanism that already makes spawn websearch-style routing work (operator: "yes, lock it in").

• Today the orchestrator "knows" a group name because before_agent_start injects ## Model Groups for spawn (names + routing rule) into the system prompt every run (index.ts, modelGroupsPromptSection), and the spawn tool description repeats the "known and confident" rule. That injection point is the mechanism to reuse.
• Main-session awareness (bullet 4): extend the same injected section to surface each group's effective modality set (text / image / reasoning), so the orchestrator routes on real capability instead of guessing from a name.
• Spawn fail-early (bullet 5): the spawn tool gains an optional requiredModalities: ("text" | "image" | "reasoning")[] parameter. resolveSpawnModelRoute validates it against the group effective set AND the routed model's actual model.input / model.reasoning; on mismatch it throws a new reason on the existing SpawnRouteError boundary — no child session is created, no third-party improvisation path.

Locked design (source of truth for implementation).

1. Effective modality set per group = the persisted override if present, else the derived common set (intersection of member input + reasoning aggregation). Overrides are capped at the member union; additions beyond that are rejected.
2. Derived at read time from the live ModelRegistry; nothing is downloaded (freshness = registry.refresh(), not a fetch).
3. Main session consumes the effective set via the same before_agent_start injection that already publishes group names.
4. Spawn detects requiredModalities at the router gate and fails early before creating a child session.
5. Pluggable: the same read-time derivation/validation pattern extends to contextWindow, maxTokens, cost (already present on each Model) without further schema changes beyond this one versioned override field.

---

Epilogue — closed interpretation gaps (A_I/A_R, 2026-08-20)

• A_I interpretation gaps: CLOSED (no open G_I); the two locked decisions (Q1 override persistence + Q2 spawn fail-early) recorded above are the operator-locked answers.
• A_R technical contract + validation plan: CLOSED (no open G_R); the contract and AC1–AC7 validation matrix were materialized in the fdw pages (session notebook). PR feat(model-groups): surface effective modalities and fail early on spawn (issue #26) #27 carries the implementation and validation results.
• This issue stays OPEN until PR feat(model-groups): surface effective modalities and fail early on spawn (issue #26) #27 lands; no auto-close.