Combine collection and sync in remote setup - #1618
Conversation
Round-1 review fixes on top of the combined collection-and-sync picker.
- The express gate's corrupt-store probe skipped the read whenever every
default row was fleet-locked. Under the old meaning ("is a named row
opted out?") that was correct; under the new one it is not, because
`source_withhold.js` throws before it filters the central ids, so a
corrupt store stops the *whole* export - and a fully fleet-managed
machine is exactly the one with no non-locked row to name. It now
reads the store on every enrolled gate, and `expressRowsSafe` stops
computing the id list nothing reads any more.
- The combined narration printed the org's rows without the
`managed by your fleet` suffix the picker and the menu both give them,
so "These will sync to your server:" read as though every row on it
were the user's to change (LLP 0188 #locked).
- The combined path dropped the `hidden_picks_syncing` span attribute
every other no-question return carries (LLP 0289 #ask-the-store).
- `pick.js` still documented the sync-scope step as where local-only is
offered, and `sync_scope.js` called its retired menu a standalone
editor with no caller. Both now say what the combined picker does.
- `docs/TEAM_SETUP.md` still walked enrolled users through three screens
including "Choose what syncs".
- `progress.test.js` hand-fed the sync lane a `Step 3 of 5` line that
`wizardStepProgress` can no longer produce, so it only asserted the
lane echoes its own input; it now pins the position-free frame.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review round 1 —
|
The combined picker's title and accept narration claimed "and sync" on a machine where confirming cannot enable it. Round 1 narrowed the express accept row for exactly that case (an unreadable client policy store: the confirm may not overwrite it, and the export seam then withholds every row), but the picker behind a Customize decline is the only screen that run decides on, and it kept the claim. Both now read the one probe. Pin round 1's fix while here: no test covered `syncWithheld` being true, so the stale `ids.length === 0` guard it removed could come back green. Two regression tests, including the fully fleet-managed machine that guard silently skipped. Also: drop a stale `@ref LLP 0338#counts-anyway` gloss on `wizardStepProgress` (the sync lane was its shipped instance and this PR retired it), correct two test names and a ref that still describe the retired "does the store withhold a named row?" trigger, and record in LLP 0396 that the policy write lands before the config commit point, so an abandoned run now leaves more shared rather than less.
|
Important Section A below is corrected by the follow-up comment. Review round 2 (of 2) at
|
| the row | reaches the store as | result |
|---|---|---|
| checked, visible, not locked | in opts.candidates, so candidateIds |
opt-out entry dropped, syncs |
| unchecked | never picked, never a candidate | existing entry preserved untouched |
| picked but display-filtered (hidden) | candidatesHiddenIds, not candidateIds |
entry preserved, and hidden_picks_syncing still computed off the pre-write read |
| org/locked | never a candidate | always syncs, entries inert at the seam |
| nothing selected | candidates.length === 0, returns before the new branch |
no write at all, same as master |
The existing.filter((entry) => !candidateIds.has(entry.source)) in
sync_scope.js:189 is the right predicate: it clears exactly the visible
selected rows and nothing else, which is what LLP 0396 says. Placing the new
branch after the candidates.length === 0 block is also right, and the
writeClientSyncEntries on an empty entries is load-bearing, not redundant:
store absence is the LLP 0188 migration marker, so materializing it is what
stops a later boot migration from deriving a withheld set for rows the user
just confirmed. Good catch on your part; the one-line comment there earns its
place.
Back navigation: the combined branch returns noQuestion: true, so
index.js's if (folders.back) { if (syncScope.noQuestion) continue atPick }
lands on the picker, as LLP 0396 requires. Verified end to end by the
back_navigation.test.js walk you updated.
Round 1's seven fixes: verified
All seven landed and are correct. On the important one (syncWithheldSafe):
the probe does now run on every enrolled gate, it is not over-eager (still
short-circuited behind enrolled(), so a solo run pays nothing), and it
cannot return a false "withheld" -- a missing file returns null without
throwing, so only a present-but-unparseable file trips it, which is exactly
the file the export seam fails closed on.
But it was incomplete in two ways, which is finding 1.
Findings
1. medium -- the sync claim was dropped from the gate but not from the picker behind it. Fixed.
src/core/cli/wizard/index.js:600, pick.js:637, pick.js:656
Round 1 made the express accept row degrade to Record everything on an
unreadable policy store, on the correct grounds that a screen may not promise
sharing the confirm cannot enable. But collectAndSync was gated only on
interactive && enrolled(), so on that same machine:
- the express accept row said
Record everything(claim withheld), and then
the pick lane's own narration immediately saidHypAware will record and sync:(claim made) -- two screens of one run contradicting each other; - a Customize run, whose picker is the only screen it decides on, was
titledWhat do you want to collect and sync?with no claim withheld
anywhere before the write.
This is a claim the PR introduces (master's title was What do you want to collect?, which promised nothing). Fixed by hoisting the probe one scope so
both claims read it:
let syncWithheld = false
...
if (interactive) {
syncWithheld = enrolled() && (await syncWithheldSafe({ opts }))
...
...(interactive && enrolled() && !syncWithheld ? { collectAndSync: true } : {}),collectAndSync only ever selects between two string literals in pick.js,
so this changes wording and nothing else; the sync lane still skips-with-warning
on that store as before. The probe now also runs on an enrolled pass that shows
no gate, because the picker makes the claim on those passes too. Cost is one
extra small-file read on an attended enrolled run.
2. medium -- round 1's central fix had no test. Fixed.
Every orchestrator-level assertion on syncWithheld asserted undefined.
Nothing pinned the true arm at all, so restoring the stale
if (ids.length === 0) return false guard left the suite green -- including on
the fully fleet-managed machine that was the whole reason for the change.
Two regression tests added in test/core/cli/wizard/index.test.js, both
mutation-checked:
- unguarding the picker's claim fails both new tests;
- re-introducing the "skip the probe when every default row is locked" behaviour
failsa fully fleet-managed machine still probes the store for its sync claim
and nothing else.
3. low -- stale @ref on wizardStepProgress. Fixed.
src/core/cli/wizard/steps.js:89
The PR correctly deletes rule 4 from the itinerary docstring, but the
@ref LLP 0338#counts-anyway below it still glossed "a lane with nothing to
ask on this machine keeps both its place in the total and its position line".
The sync lane was that rule's only shipped instance and this PR retires it;
progress.test.js now asserts the opposite for it. Re-glossed to what the
function still implements, naming LLP 0396 as what retired the instance.
(@ref LLP 0338#consequences at line 70 still holds and is untouched.)
4. low -- two test names and a ref describing the retired trigger. Fixed.
index.test.js:445was still titled "a standing opt-out ... narrows the
express gate's sync claim" over an assertion that it does not, with a lead
comment stating the retired rule as fact.express.test.js:119was titled and glossed as "the store already withholds
one of the named rows", which round 1 replaced with "the store is unreadable",
and carried@ref LLP 0201#gatewhereexpress.jsnow carries LLP 0396.
Both retitled and re-glossed. The assertions were already correct; only the
prose lied.
5. low -- sync_scope.js:28 says the menu "stands unchanged for direct
callers and its own tests". Not fixed, reported. There are no direct callers
outside tests: src/core/cli/wizard/index.js is the only production caller and
it passes collectAndSync: true unconditionally. See item C.
The three items round 1 left open
A. The policy write precedes the commit point. medium. Left, and now recorded in the LLP.
Correct that the write position is master's and not a regression. But the
substance that is new is the direction of the leftover. On master an
abandoned run left extra opt-outs standing: it failed closed, more withheld
than the user settled on. Here an abandoned run leaves opt-outs cleared: it
fails open, more shared than the user settled on, and neither Back nor Ctrl+C
restores them. A Back-then-uncheck also cannot put back an opt-out the first
pass destroyed.
I did not fix it, on purpose. Making the clearing recoverable means the lane
returning "clear these" and the orchestrator writing after
commitWizardPickedConfig, which changes the lane's contract for its ~30
direct-calling tests. That is a redesign of your feature at the review cap, not
the smallest change, and the fail-open direction is a call for the maintainer,
not for me.
What I did do: added a paragraph to llp/0396 (still Draft, so still
editable) recording the ordering and naming the direction change explicitly, so
the next reader meets it in the spec instead of deriving it from index.js.
Please read that paragraph and push back if I have characterised your intent
wrongly.
Maintainer: this is the one thing in the PR I would want a human decision on
before merge.
B. No per-row sign that a checked source is currently local-only. medium. Left; it is the design.
I agree with round 1 that this is your call, and I want to say why more
strongly than "it is documented". One checkbox carrying two axes is the point
of the PR; a picker able to round-trip "collect but do not sync" would need the
second axis back, which is the thing being retired. LLP 0396 states the clearing,
docs/TEAM_SETUP.md now warns about it on exactly the path where it bites
(Customize), and the lane narrates the resulting list.
One thing worth having on the record: sync_scope.js's own docstring still
describes the property being given up, that "a re-entry renders the sources
already opted out unchecked so re-running the wizard round-trips the store
instead of resetting it". That was an explicit invariant, and the combined
picker deliberately drops it. buildPickOption's docstring names the mechanism
a future change would revive (the retired · stays on this machine suffix), so
the way back is documented if the maintainer wants it. Not churning it at the cap.
C. The sync menu is unreachable in production. low. Left.
Confirmed by grep: index.js:659 is the only production caller and it passes
collectAndSync: true unconditionally, so both promptSyncScopeSelection and
the autoAccept narration arm above it are dead in production. Deleting takes
~25 tests, SYNC_SCOPE_MENU_TITLE, WIZARD_STEP_LABELS.sync, the 'sync'
member of WizardStepName, and the enterKeepsChecked coverage with it. That
is a separate change; CLAUDE.md says land the small one and defer the rest, and
the file's own docstring already discloses the situation. Not filing an issue
for it either: it is maintenance drag with no user-facing consequence, which is
below the bar for autonomous issue-filing. Recorded here so it is not lost.
CPU and memory pass
No CPU or memory concern. Nothing here is a hot path, a per-record path, or
a long-running process: this is the interactive setup wizard, bounded by the
picker row count and the policy entry count (single digits each), running once
per install or reconfigure.
syncWithheldSafeis onereadObservabilityEnvplus one small-file read and
parse. Round 1 made it strictly cheaper than master's version, which also
built aSetfromoptedOutClientSourceIdsand scannedidsagainst it;
the entries are now read and discarded. My change adds at most one such read
peratExpresspass (so one more per back-navigation), still behind
enrolled(). I annotated the read-and-discard so it does not read as an
accidental unused result.expressRowsSafedropped a secondfilter().map()overdefaultRowsand its
wrapper object. Less work and less allocation than master.- The
collectAndSyncbranch insync_scope.jsis one O(entries) filter plus
onewriteClientSyncEntries(aMapplus a sort over single digits) and one
atomic write, replacing master's prompt plus the same write. It writes even
when the content is unchanged, which is once per run and load-bearing for the
migration marker. wizardItinerarystill allocates one array copy per call, unchanged, and the
itineraries got shorter (5 to 4, 4 to 3), soindexOfscans marginally
shorter.wizardStepProgressadded one string comparison.pick.jsadded two ternaries over string literals: no allocation change.
No new dependencies, no unbounded growth, no busy loops, nothing that worsens
with data volume or uptime.
Conventions
No semicolons, no U+2014, no NUL bytes, no @typedef, no inline import('...')
types, root-anchored .js type-import specifiers in the touched types.d.ts
imports, no new runtime dependencies. All clean, in your changes and in mine.
Marked findings rather than clean: items A and B are real and unresolved by
design, and A in particular is a privacy-direction change that needs a human's
yes before this merges. Nothing I consider a defect remains.
Peer review caught that the lane which now *clears* standing local-only
opt-outs is the one lane that says nothing about it. The menu it replaced
printed the mirror line whenever it kept one ("Keeping local-only: X.
Change later with 'hyp privacy client ...'"), and `hyp privacy client
<name> sync` prints the same two qualifiers for the identical store
write: the flip is future-only, and there is a command back.
The "These will sync to your server:" list cannot carry it, because a row
reads the same there whether it was already syncing or was local-only
until this keypress. So state the revocation separately, or not at all
when nothing was revoked.
Also record `sources_cleared` on wizard.sync_scope.finish:
`sources_opted_out` is 0 on every combined run by construction, so
without it a "setup turned my sync back on" report has no signal behind
it at all.
This does not fix the ordering: the write still lands before the config
commit point, so a cancel at the folder question still leaves the
revocation standing. That is reported as high and left for the
maintainer. It does mean the user is told it happened.
Correction to my review above, and six further findingsA second reviewer ran the round in parallel and refuted a premise I relied on. The correction: the write ordering is a new regressionI wrote that "the write position is master's and not a regression". The position master ( this PR ( The opt-out is gone, no config was committed, and nothing was printed on Master's old auto-accept arm returned
I still have not fixed the ordering, and I still think that is right at the What I did fix (
|
Neutral is stuck on this PR and needs a human decisionWhat neutral was doing: triage at head First: this is good work. The combined picker's mapping is correct on every path two review rounds could construct, the LLP extensions are genuine, and both rounds' fixes landed cleanly. One ordering issue is the only thing standing between this PR and merge. The blocker: a cancelled run destroys a standing privacy opt-outRound 2's correction claimed it; neutral re-derived it independently before acting on it. Probe: enrolled reconfigure, policy store holding
Mechanism: the combined arm in That is a user losing a privacy setting they never chose to change, failing open (more shared than they settled on). Neutral classifies it as blocking and will not resolve a fail-open privacy direction on its own. The decision neededOne of:
Neutral's own read is that option 1 matches the repo's stated transaction shape (LLP 0190 #commit-point: every question lane runs, then the run commits), but the call belongs to the author and maintainer. Non-blocking residuals, so you see the whole pictureNone of these block; they are recorded in the review thread and can ride or be deferred once the blocker is decided:
How to unstickReply on this PR with your decision, or push the fix to |
|
Fixed the blocking write-ordering issue in 698d09b. The combined sync lane now returns pending source ids without changing the policy store. The orchestrator applies only the final selection after the config commit succeeds. Back discards the pending selection; Cancel, overwrite refusal, and config-write failure preserve existing opt-outs. At commit, the policy store is re-read so unrelated changes made while setup was open are preserved. An unreadable store or failed policy save stops setup without clearing it. Revocation narration and the sources_cleared commit signal follow the successful write. The config may already have landed if the policy save fails; the two saves are ordered, not an atomic transaction. Regression coverage exercises Cancel, Back-then-uncheck, overwrite refusal, config-save failure, policy-save failure, newly corrupted policy data, and successful commit preserving an unrelated opt-out. LLP 0396 and the team setup guide now describe the deferred behavior. Validation: all 366 affected tests, type checking, and the onboarding smoke pass. Full suite: 6517 passed, 3 skipped, and the previously identified unrelated mcp-stdio-write-backstop JSON.stringify depth test failed. CPU and memory review found no concerns: processing remains bounded by the selected sources and existing policy entries. |
Remote setup currently asks what to collect, then asks again which selected sources to sync. Combine these into one "What do you want to collect and sync?" selection, with matching progress counts and Back navigation from folder handling.
Confirming a visible selected source enables both collection and sync, including clearing its existing client opt-out. Unselected and hidden source policies, locked fleet sources, directory policies, and first-sync holds retain their existing behavior. Unreadable policy stores remain intact and exports fail closed. LLP 0396 records the updated setup contract.
Validation: 355 affected tests pass, type checking passes, and the hermetic walkthrough_picker_to_first_query smoke passes. The full suite exposed an unrelated mcp-stdio-write-backstop JSON.stringify depth test failure, which also reproduces in isolation.
CPU and memory review: no concerns. Work is bounded by the existing picker and policy entry counts, with no new dependencies or background activity.