From 045b39d68612ce3ba9f8f93544f1aaa213be4bfb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Val=20Alexander=20=F0=9F=91=91?= Date: Thu, 12 Nov 2026 10:48:46 -0600 Subject: [PATCH 1/2] docs: reconcile delivered local-model status and qualification gaps --- docs/design/local-models-settings.md | 8 ++++---- docs/status.md | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/design/local-models-settings.md b/docs/design/local-models-settings.md index eea646ff..0e51cb6c 100644 --- a/docs/design/local-models-settings.md +++ b/docs/design/local-models-settings.md @@ -1,6 +1,6 @@ # Settings → Local models: design brief for LM-4 (#381) -Status: brief only, 2026-11-03. No UI has been built. Written against `main` at `b785054` and the producers there. It doesn't change producer policy. Two controls wait on Val's answers in [the decision record](../reviews/2026-11-03-lm5-val-decision.md). +Status: partially implemented on protected main through #498, #499, #502 and #503, checked 2026-11-12. Settings connection management, metadata discovery, model-menu filtering and draft/focus preservation are implemented. The explicit selected-model native compatibility test and configuration-bound qualification-result workflow remain unimplemented; the qualification labels below describe the required design, not a currently available tested state. Product execution remains unadmitted. The original 2026-11-03 brief was written against `b785054`; its [decision record](../reviews/2026-11-03-lm5-val-decision.md) remains dated history. ## What a person can do @@ -63,10 +63,10 @@ While a target is set: **Cold load (LM-2 step 6).** Beside the model list: "Checking models doesn't load anything. Your first message may make the server load the model, which can take a while and use a lot of memory." -## Waiting on Val +## Delivered policy controls -- **Clearing to cloud.** Recommended: a confirmation naming the cloud model. Build the plain control first, behind that answer. -- **Remote clients.** Phone and paired desktop show the inference host and can't change the target. Whether they can send at all is the open decision; design the read-only display either way. Older clients get readable text and no controls. +- **Clearing to cloud.** The delivered UI asks for confirmation naming the cloud model. This is an explicit destination change; it does not admit local execution. +- **Remote clients.** Main refuses phone and paired-desktop sends on local threads, and only this Mac’s window may choose, change or clear the target. Complete paired host/model presentation and older-client acceptance remain separate requirements; they are not established by this send refusal. ## Tests diff --git a/docs/status.md b/docs/status.md index 0f624985..12c89581 100644 --- a/docs/status.md +++ b/docs/status.md @@ -151,12 +151,12 @@ before relying on a tick. - **Actionable local context failures** (#380, #498). The captured real native dense-input run failed after compaction at 73,800 tokens against a 65,536-token slot; the outbound request retained all three prior dense inputs and the fourth new input. The repair explains recognized context-limit failures while preserving failed status, native diagnostics and no automatic retry/model switch. Build/typechecks passed; final full units passed with 2,435 passes, 36 explicit skips and zero failures. Four installed-Codex/scripted-server failure cases passed, each with one synthetic command effect and no repeated request or model change. The first final unit run's Vite port failure remains retained despite a green explicit revalidation. Parent `da13074` passed Windows, but Linux package-host DNS and initial paired-iPhone qualification failed; archived TLS activity reaches Security/trustd before the request timeout, with no cause or trust repair established. Fresh exact-head hosted checks and all admission/native/human gates remain open. [Evidence and limits](reviews/2026-11-11-local-context-overflow.md). -- **Local draft destination fence** (#380, #381, #498). Final review found that an asynchronous local selection could accept input/send through its old non-local configuration before the target was pinned. Two controlled regressions failed; the repair reserves and fences only the source/new target, preserves the exact raw draft, and leaves unrelated Stop and editing available. All three focused Electron cases and full frozen build/typechecks/units pass (2,465 unit passes, 36 skips), followed by 511 desktop E2E passes with 11 skips and seven site passes. Fresh hosted qualification and protected delivery remain required. Product local execution remains unadmitted. [Failure, repair and limits](reviews/2026-11-11-local-selection-fence.md). +- **Local draft destination fence** (#380, #381, #498). Final review found that an asynchronous local selection could accept input/send through its old non-local configuration before the target was pinned. Two controlled regressions failed; the repair reserves and fences only the source/new target, preserves the exact raw draft, and leaves unrelated Stop and editing available. All three focused Electron cases and full frozen build/typechecks/units pass (2,465 unit passes, 36 skips), followed by 511 desktop E2E passes with 11 skips and seven site passes. Protected [#498](https://github.com/TypeSafeAI/modex/pull/498) has since delivered this repair; its exact merge-group macOS check passed. The counts above retain their original candidate scope, and product local execution remains unadmitted. [Failure, repair and limits](reviews/2026-11-11-local-selection-fence.md). - **iPhone image-binding diagnostics** (#109, #498). The prior manifest collapsed container-query failures, realpath failures and image mismatches into app-image refusals. A diagnostic-only increment retains at most 16 callback/time-bound outcomes with sanitized error metadata and unchanged identity/command/stop guards. Six baseline controls fail; all 25 observer cases pass after the change. Full frozen build/typechecks/units pass with 2,471 passes, 36 skips and zero failures/cancellations. Fresh hosted observation remains required; no successful stack binding or older TLS cause is claimed. [Evidence and limits](reviews/2026-11-11-ios-image-binding-diagnostics.md). - **iPhone trust observer collection** (#498). Published `188af74` passed 44 iPhone tests with zero actual failures, but its observer stopped after six poll errors and unconfirmed command closure, with no callbacks/stacks captured. A predicate-only experiment narrows app/Security and trustd-peer logs while keeping every bound and identity guard. Native Foundation predicate replay goes from output-limit failure to 21 selected records with preserved app/trustd correlation; all 19 focused tests pass. Full build/typechecks and units passed with 2,465 passes, 36 skips and zero failures/cancellations. Fresh `aa095b2` app qualification passes 44 tests, but nine image-binding refusal entries yielded zero bound observers/stacks; five distinct callbacks in offline diagnostics completed normally. Live stack binding remains unqualified and the older trust-stall cause remains unknown. [Evidence and limits](reviews/2026-11-11-ios-trust-predicate.md). - **Code Review native-modal investigation** (#498). Parent `fc593e0` timed out clicking Settings before Jev contrast collection; teardown then timed out and the job reached its 45-minute limit. The retained Electron stack is inside `NSAlert runModal`, with dialog message and cause unknown. A test-only candidate records bounded owned-child stderr and observes uncaught exceptions without handling them; five real-child controls and all eight accessibility cases pass locally. Full build/typechecks and units passed with 2,460 passes, 36 skips and zero failures/cancellations. Exact-head hosted qualification remains required; no causal repair or acceptance is claimed. [Observation and limits](reviews/2026-11-11-electron-modal-observation.md). - **Linux Debian source continuity** (#498). Candidate `188af74` acquired all four signed Noble pockets and installed host dependencies, then failed baseline Debian installation because `libnotify4` was not installable. The next candidate keeps explicit signed sources through both baseline and upgrade dependency resolution, preserving artifact verification, installed checks and the three-minute direct command bound. All 13 synthetic helper/wrapper cases and final corrected-source build/typechecks/units pass, with 2,465 passes, 36 skips and zero failures/cancellations. Fresh `aa095b2` hosted installed checks now pass eight Linux launches plus controlled KWallet5 and three Windows NSIS/DPAPI launches; original installer acceptance remains pending, including historical-schema, full graphical-login and real-account limits. [Failure, candidate and limits](reviews/2026-11-11-linux-debian-sources.md). -- **Linux qualification host preparation** (#498). Candidate `5edbb84` passed required hosted macOS, iPhone and Windows checks, but both Linux attempts stopped at the 30-minute job limit during Azure Ubuntu mirror DNS failures before installed-app acceptance. The CI-only repair selects official signed Noble archive/security sources per APT invocation, bounds acquisition and the host step, and preserves the nine packages and downstream installed acceptance. Build passed; final full units passed with 2,427 passes, 35 skips and zero failures. Seven focused helper cases and independent review cover argument boundaries, failure propagation and owned temporary cleanup. Application/native inputs are unchanged. Fresh exact-head hosted qualification remains required; no Linux acceptance or product admission is claimed. [Failure evidence, repair and limits](reviews/2026-11-11-linux-host-preparation.md). +- **Linux qualification host preparation** (#498). Candidate `5edbb84` passed required hosted macOS, iPhone and Windows checks, but both Linux attempts stopped at the 30-minute job limit during Azure Ubuntu mirror DNS failures before installed-app acceptance. The CI-only repair selects official signed Noble archive/security sources per APT invocation, bounds acquisition and the host step, and preserves the nine packages and downstream installed acceptance. Build passed; final full units passed with 2,427 passes, 35 skips and zero failures. Seven focused helper cases and independent review cover argument boundaries, failure propagation and owned temporary cleanup. Application/native inputs were unchanged by that CI-only repair, which has since landed through protected [#498](https://github.com/TypeSafeAI/modex/pull/498). Its later delivery is distinct from the historical host failures above; native account/graphical-session acceptance and product admission remain open. [Failure evidence, repair and limits](reviews/2026-11-11-linux-host-preparation.md). - **Companion command recovery (COMP-CMD-2).** #497 is on main as `85143ee`; its tree exactly matches reviewed `d3d2eee`. It keeps timed-out command reads in the picker with explicit retry, retained search/drafts and keyboard Done actions. The required protected queue check passed 505 desktop cases (four skips) and seven site cases; iPhone was skipped for that merge-group event. The independent three-cycle context regression passes without #496's reset, so #496 is closed as superseded; its branch and checkout are preserved. Two fresh-install closeout integration suites each passed 44 executions (43 ordinary passes and one intentional expected failure, zero actual failures/skips), with source hashes unchanged. Human and physical-device acceptance on #276/#281 remains open. [Current integration qualification](reviews/2026-11-11-native-closeout-delivery.md); [historical producer review](reviews/2026-11-08-companion-recovery-pr.md). - **v0.4.2 release.** #492 (`4297235`) prepared it with Code Review hidden by default. Its tag exists, and the GitHub release remains a draft; it is not published. [Review](reviews/2026-11-03-v0.4.2-release-review.md). @@ -166,7 +166,7 @@ before relying on a tick. - **Private local workspace continuation** (#379/#380, #498). A network-denied real Codex 0.167.0/Ollama 0.40.16 read reproduced refusal on the next turn after the child persisted its own workspace trust mark. The strict repair admits only that one main-bound private workspace entry, keeps other settings and managed/project overrides refused, and checks captured realpaths before setup and after configuration reads. These spot checks do not establish race-free filesystem identity. All 45 focused tests, root build/typechecks and full units (2,420 passes, 35 skips) passed on the frozen candidate; full E2E passed with 508 desktop passes, 11 explicit skips and seven site passes, with no failures or observed retries. The unchanged 500 ms cold-open gate measured 73.2 ms; all four affected real-model Electron lifecycle cases separately passed with zero skips or retries. Native continuation and completed compaction were observed, but independent session audit found two repeated reads after compaction, so action-safe compaction and production admission remain unqualified. [New qualification and limits](reviews/2026-11-11-native-private-workspace-trust.md). - **Local models** ([tracker #376](https://github.com/TypeSafeAI/modex/issues/376)). Main holds LM-1's contract, LM-2's loopback discovery, LM-3's Codex execution and LM-5's refusal boundaries, but does not construct `LocalExecution`, so a built app refuses local execution. Protected #498 delivered the `issue-closeout` integration at `5f1f1fe8bfa93ee971de7b443f794335ae9c414e`, preserving the `lm-v1-workflow` and `lm-native-recovery` stacks, `lm-ready` (`01101cb`), `lm5-fixtures-rebased` (`7a38090`) and Val-authorized frozen UI/draft-recovery snapshots without changing their owners' work. Connection management, revision-aware model/context selection, destination display, main-owned paired-send refusal and cloud-return confirmation are now on main. The final protected queue passed 2,471 units with 36 skips, 514 desktop E2E cases with eight skips and seven site cases. Admission branch `lm5-admission` (`1276a29`) remains unapplied; remaining product decisions and activation retain their separate gates. [How it works](local-models.md#lm-3-local-execution-through-codex). - The integration repairs packaged credential storage selection under inherited test flags and exposes the proposed follow-up text in its accessible button name. At executable candidate `7d6094e`, build/typecheck, 2,407 units (35 skips), 508 desktop E2E cases (11 opt-in skips), seven site cases and four focused Local Models journeys passed. Separate CI opt-ins passed three Open Knowledge Electron cases and one unit. Four real-model Electron lifecycle cases and the network-denied read/edit/check canary passed on Codex 0.167.0/Ollama 0.40.16 with the approved model digest, observed 65,536-token context and unchanged personal configuration. The original offline workspace-placement failure is retained. Packaged Keychain acceptance still needs native interaction after a blocked first attempt; production execution and human acceptance remain unadmitted. [Current qualification, memory comparison, failures and limits](reviews/2026-11-11-closeout-qualification.md). + The integration repairs packaged credential storage selection under inherited test flags and exposes the proposed follow-up text in its accessible button name. At executable candidate `7d6094e`, build/typecheck, 2,407 units (35 skips), 508 desktop E2E cases (11 opt-in skips), seven site cases and four focused Local Models journeys passed. Separate CI opt-ins passed three Open Knowledge Electron cases and one unit. Four real-model Electron lifecycle cases and the network-denied read/edit/check canary passed on Codex 0.167.0/Ollama 0.40.16 with the approved model digest, observed 65,536-token context and unchanged personal configuration. The original offline workspace-placement failure is retained. The later [packaged macOS synthetic-key receipt](https://github.com/TypeSafeAI/modex/issues/378#issuecomment-6822708976) verifies OS-backed encryption, masking, retrieval and clearing on a private ad-hoc arm64 QA app with application inputs equal to protected `558b54f8`. Its first Settings-click timeout and Val’s report that no Keychain prompt appeared remain retained; one recorded second invocation passed the unchanged assertions. This does not establish a cold-start repair, installed release or universal platform behavior; production execution and human acceptance remain unadmitted. [Current qualification, memory comparison, failures and limits](reviews/2026-11-11-closeout-qualification.md). Real tokenless discovery through `LocalModelsService` observed exactly GET `/api/tags`, GET `/api/ps` and POST `/api/show` on the configured loopback Ollama, with no inference or credential use. Its advertised 262,144-token context is distinct from the separately observed 65,536 loaded context. The canonical token-record suite now also proves actual Save to Notes/reopen exclusion with no extra inference. These candidate observations do not qualify OS crypto, UI/main-IPC authority, other runtimes or product admission. [Native delivery and metadata qualification](reviews/2026-11-11-native-closeout-delivery.md). @@ -179,7 +179,7 @@ before relying on a tick. - **Real-model Electron lifecycle qualification** (#380 AC4 / #383 AC1). Signed implementation `c72a956` on the preserved `lm-native-recovery` branch qualifies real Electron/main/Codex/Ollama read/edit/check/diff, Stop and owned Codex/server loss, with stale-handle refusal and exact dispatch/effect counts. Regressions first reproduced and then repaired local native reconnect/quiet-final handling and a redundant acknowledged-image-draft save that vetoed Quit. On exact tree `c7e87f8`, build and typecheck passed; unit tests had 2,403 passes and 35 skips; desktop E2E had 503 passes and 11 skips; site E2E had seven passes; all four real-model lifecycle cases and the separate network-denied canary passed. All eight Images cases passed in the complete suite, retaining delayed/rejected-save protection. Independent audits verified actual bytes, commands and effects. The existing-key signing gate is resolved, its signature verifies and a complete-history bundle restores the signed implementation exactly; later evidence/status edits change documentation only. Both failed complete runs, signing timeouts and the model-prose failure remain retained. Production execution, issue checkboxes, human/paired acceptance and publication remain unchanged. [Protocol, failures and limits](reviews/2026-11-08-local-model-native-recovery.md). - **Claude account pool live acceptance.** The pool landed in #467. No live OrcaRouter turn, second subscription or real usage-limit message has run. [Design and limits](claude-accounts.md), [repair evidence](reviews/2026-11-01-claude-pool-repair.md). -- **Existing PR reconciliation.** #490 carries the content of #481, #484 and #485. #486 was closed on 2026-11-11 after verifying that protected merge #488 replaced it with the safer `retire`; its original worktree and branch were preserved. #475 still has its pre-rebase head; its fixtures are included in the integration below, pending current-source qualification and protected delivery. +- **Existing PR reconciliation.** #490 carries the content of #481, #484 and #485. #486 was closed on 2026-11-11 after verifying that protected merge #488 replaced it with the safer `retire`; its original worktree and branch were preserved. Owner PR #475 remains open at `7cc57139`. The rebased fixture stack was included in protected integration #498; that delivery does not complete #380’s real compaction/privacy/resource acceptance or dispose of the owner’s PR and checkout. - **Companion network acceptance (#109).** The [two-LAN evidence map](reviews/2026-10-28-companion-two-lan-acceptance.md) ties each of the four #109 requirements to the tests that prove it, adds the last race (the interface list changes while Forget paired phones or Turn off runs, and no listener survives with the old token; both shown red against a mutation, green on `main`), and holds the physical two-subnet procedure. Two gates remain with Val: running that procedure, and deciding whether VPN tunnels and virtual bridges with private addresses get a listener ([question](https://github.com/TypeSafeAI/modex/issues/109#issuecomment-6464762470)). The filter is unchanged until Val answers. - **v0.4.0 post-publication acceptance and Companion 0.4.0 (9).** Desktop publication is complete. v0.3.0's two post-publication gates (the banner upgrade of the release Mac and the Companion 0.3.0 (8) upload) are superseded: the installed Mac reports signed 0.4.0, and Companion build 9 replaces build 8. The v0.3.0-to-v0.4.0 background upgrade remains unverified: the installed Mac already reports signed 0.4.0 and was left unchanged. Companion build 9 has not been delivered by this release session; its TestFlight upload and physical-device checks remain separate. Human accessibility, real-provider and native-platform account/session checks retain their existing gates. [Publication evidence and limits](reviews/2026-10-24-v0.4.0-release-review.md#published--2026-10-30-utc). - **Mac App Store edition.** A sandboxed Store front end with a separately installed signed From 3adc14f8436512013291f4fd113650fbf019f001 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Val=20Alexander=20=F0=9F=91=91?= Date: Thu, 12 Nov 2026 11:03:58 -0600 Subject: [PATCH 2/2] docs: record accepted CR-8 fixture journey --- docs/status.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/status.md b/docs/status.md index 12c89581..e81f10fb 100644 --- a/docs/status.md +++ b/docs/status.md @@ -130,7 +130,7 @@ before relying on a tick. | [#324](https://github.com/TypeSafeAI/modex/issues/324) CR-5 | 12/14 | Original R4 comparison (AC1), retired on 2026-11-04 but still named by the box; the Viewed toggle is decided (local, revision-bound) and not built | Build the Viewed toggle; run the benchmark | | [#325](https://github.com/TypeSafeAI/modex/issues/325) CR-6 | 13/14 | Authorized live native smoke (AC6) | Run the smoke once on a small open TypeSafeAI/modex PR (authorized 2026-11-04) | | [#326](https://github.com/TypeSafeAI/modex/issues/326) CR-7 | 13/14 | Live canary on a disposable repository (AC6) | Val names a disposable repository | -| [#327](https://github.com/TypeSafeAI/modex/issues/327) CR-8 | 5/17 | Reference comparison (retired 2026-11-04, boxes still name it), VoiceOver and the live gates. Budgets and C76 landed in #490; re-check item 18 and AC5 once Val confirms the budget numbers | Val's [one session](reviews/2026-11-02-v0.5.0-val-session.md) | +| [#327](https://github.com/TypeSafeAI/modex/issues/327) CR-8 | 6/17 | The full fixture journey is accepted at its [named source/CI scope](https://github.com/TypeSafeAI/modex/issues/327#issuecomment-6830284937); reference comparison (retired 2026-11-04, boxes still name it), VoiceOver and the live gates. Budgets and C76 landed in #490; re-check item 18 and AC5 once Val confirms the budget numbers | Val's [one session](reviews/2026-11-02-v0.5.0-val-session.md) | | [#376](https://github.com/TypeSafeAI/modex/issues/376) LM tracker | 1/14 | LM-2 to LM-7 core acceptance; LM-8 is optional and does not block the core release; `LocalExecution` is not constructed, so the product refuses local turns | Val answers the five questions in `docs/reviews/2026-11-03-lm5-val-decision.md` on branch `lm-ready` | | [#378](https://github.com/TypeSafeAI/modex/issues/378) LM-2 | 12/13 | #498 delivered the cold-load explanation and token/export regressions; #499 delivered manual-ID visibility and the complete metadata IPC fixture matrix. Packaged secure-storage acceptance remains open | Complete secure-storage acceptance and re-audit every criterion; existing ticks do not replace current evidence. [Receipt](reviews/2026-11-12-lm2-manual-visibility.md) | | [#379](https://github.com/TypeSafeAI/modex/issues/379) LM-3 | 12/13 | AC1's real native read/edit/check and AC4's regression/configuration receipts now bind to actual main. AC6 still has native redirect and managed-policy gaps; simultaneous isolation claims retain their tested fixture scope | Resolve AC6 without weakening destination binding; product admission stays separate. [Receipt](https://github.com/TypeSafeAI/modex/issues/379#issuecomment-6811858808) |