Measure copilot and omp first-run surfaces, and commit the captures - #157
Merged
Conversation
The 2026-09-09 record closed this question for claude and codex and named its own gap: nothing about copilot or omp, which were not exercised. This closes it by measurement. Neither vendor behaves the way the claude and codex pattern predicts. Copilot raises a folder-trust dialog whose focused default is "Yes", so an unattended Enter grants read access and permission-gated write and shell execution in whatever directory the worker was pointed at. That is the same hazard already measured on codex, in a vendor currently outside the park-and-escalate path. omp raises no trust dialog at all -- the string does not occur in any captured frame. It raises a five-step first-run setup wizard that blocks the composer, whose first step confirms a provider sign-in on Enter. Per the maintainer's ruling that is not a gate crew escalates on: crew never launches omp under a fresh home, so the wizard means the machine was never configured. omp's predicate recognises the normal prompt and returns PromptReady; the wizard stays Undecided and fails the start closed with a typed error. Step 1 is committed as the negative sample that must classify that way; steps 2 to 5 are observations in the record, not fixtures. The grid gains handling for four sequences the new captures are the first to emit. Window manipulation and DECRQM mode queries join the no-op arm -- they are answered by the terminal, not by the screen, and the replies are what the viewer-socket filter already discards, so both halves of that exchange are accounted for. CSI 1 K and CSI 3 J are implemented rather than ignored: the first erases the line start through the cursor inclusive, the second erases scrollback this fixed-height model does not have and must therefore leave the visible screen standing. Each has a test confirmed to fail against the opposite behaviour. Three method findings are in the record because they cost real time. `startup.showSplash` is not what animates omp's first paint and setting it made the capture marginally larger. A settle detector that treats "silent because the vendor has not begun painting" as "silent because it has finished" captures the whole animated startup and looks plausible doing it. And a phrase read out of escape-stripped text is a hypothesis: an early reading reported a legend that does not exist in the bytes and would have inverted the security conclusion for step 1. Two properties of the fixtures worth keeping. No committed capture contains an absolute home path -- copilot's dialog prints the directory it asks about, so the first recording carried one, and every probe was re-run from a neutral scratch directory. And both vendors self-updated during the session, so each capture that can carry its own version string in-band now does, and can be pinned from the bytes rather than from prose.
The vendor-documentation section carried this as intended-but-unverified: copilot's changelog claims a linked worktree inherits the main checkout's trust, but the resolution lives in a compiled module, so reading could not settle it. It is now measured. A real git repository with a `git worktree add` worktree, under a throwaway home: granting trust in the main checkout leaves the worktree launch with no dialog, and the config gains one entry naming the repository root as a realpath. Nothing is written naming the worktree. The control is the only reason this is not the opposite finding. The first attempt selected the trust option on a timer, the keystroke never registered, Enter took the session-only option, and the worktree then showed a dialog -- which reads exactly like "worktrees do not inherit trust", with a clean capture behind it. Re-launching in the main checkout first is what revealed there was no persisted trust to inherit. The harness now refuses to press Enter until the selection marker is verifiably on the intended option, rather than sending it on a delay. A keystroke is not an event that can be assumed to have landed; the capture has to show the screen responding before the next key is sent. Same shape as the settle-detector failure already recorded here.
The Limits section said neither vendor's post-setup screen had been reached and that the phrase-absence claim was therefore unmeasured. That bullet predated the composer captures and, once omp-composer.raw was committed, contradicted a fixture sitting beside it in the same change. Both normal screens were in fact reached. omp's composer came from a sixth Esc past the last wizard step; copilot's came from the two no-dialog launches of the worktree-inheritance check. So the claim is now measured rather than hedged: omp's wizard constants occur zero times in omp's composer, and copilot's dialog phrases occur zero times in either copilot post-trust capture, all counted as literal bytes. Committing copilot's composer as a fixture rather than only citing it. Copilot's predicate needs a screen it must classify as ready for the same reason omp's does, and an exclusivity claim that can only be checked against uncommitted captures is not one a reviewer can check at all. The sentence that a false positive on a normal screen parks a ready run rather than writing into a gate is kept: the failure direction has not changed, only the amount of evidence behind it.
`copilot-composer.raw` was captured under a throwaway home whose account is unauthenticated, and its status line says so. The screen is a genuine composer -- input area, tab bar and footer all present -- but a signed-in composer's exact rendering is unmeasured, and the record did not say so. The consequence is directional and worth stating rather than leaving for someone to discover: a predicate keyed on the status line, or on anything else auth-shaped, would fail closed on every normal signed-in machine, which is the common case. That is the opposite failure direction from the one this work exists to prevent, and just as real. So a copilot predicate must key on chrome the composer shows regardless of sign-in state, and say in its test which bytes it keys on and that the status line is excluded. The signed-in rendering is a verification item rather than a fixture gap: copilot runs in the supervised end-to-end exercise, where a signed-in ready screen is observed for free -- cheaper and more honest than another probe under a real home. The README row carries the same warning where someone reaching for the fixture will actually read it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 2026-09-09 record closed the first-run-gate question for claude and codex and named its own gap: "none about copilot or omp-rpc, which were not exercised." This closes it by measurement. Neither vendor behaves the way the claude/codex pattern predicts.
Copilot raises a trust gate focused on "Yes"
An unattended Enter grants read access and permission-gated write and shell execution in whatever directory the worker was pointed at — the same hazard already measured on codex, in a vendor currently outside the park-and-escalate path.
omp raises no trust gate at all
The string does not occur in any captured frame. It raises a five-step first-run setup wizard that blocks the composer, whose first step confirms a provider sign-in on Enter.
Per the maintainer's ruling that is not a gate crew escalates on: crew never launches omp under a fresh home, so the wizard means the machine was never configured. omp's predicate recognises the normal prompt and returns
PromptReady; the wizard staysUndecidedand fails the start closed with a typed error. Step 1 is committed as the negative sample that must classify that way; steps 2–5 are observations in the record, not fixtures.Three fixtures
copilot-folder-trust.rawomp-setup-step1.rawUndecidedomp-composer.rawPromptReadyThe grid needed four new sequences, and the fixtures are what licensed adding them
The grid panics in test builds on an unknown sequence, with a message demanding the fixture arrive first. It did.
Window manipulation (
CSI 22;0 t) and DECRQM mode queries (CSI ? Ps $ p) join the no-op arm — they are answered by the terminal, not by the screen, and the replies are what the viewer-socket filter already discards, so both halves of that exchange are accounted for.Two are not no-ops and are implemented:
CSI 1 Kerases the line start through the cursor inclusive. The off-by-one matters: excluding the cursor cell leaves exactly the character the sequence was sent to clear.CSI 3 Jerases scrollback. This grid is a fixed 40-row window with no scrollback structure, so it must leave the visible screen standing — a model that cleared the screen would report an empty surface for a populated one.Each has a test confirmed to fail against the opposite behaviour (
..=flipped to..;3Jmade to clear the screen).Three method findings, because they cost real time
startup.showSplashis not what animates omp's first paint. Setting it made the capture marginally larger.Two properties of the fixtures worth keeping
No committed capture contains an absolute home path. Copilot's dialog prints the directory it asks about, so the first recording carried one, and every pre-existing fixture has zero. Every probe was re-run from a neutral
/tmpscratch. No text rule catches this class, becausefixtures/is exempt from the marker guard by design.Both vendors self-updated mid-session (copilot 1.0.81 → 1.0.83, omp 18.1.15 → 18.1.16). Each capture that can carry its version string in-band now does, so it can be pinned from the bytes rather than from prose.
omp-setup-step1.rawcannot, and the record says so.Gate
cargo test --workspace69 suites, exit 0 ·cargo fmt --all --check·clippy -D warnings·check:markers432 scanned, 0 findings ·format:check·typecheck·check-trailersclean.