Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/lock-identity-guard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"devctl": minor
---

`devctl lock` can now tell you when a command changed the state it was guarding while a server still held that state open. A `locks` entry may name where the resource lives on disk (`{"name": "d1", "path": ".wrangler/state/v3/d1"}`, alongside the plain `"d1"` form, which keeps working unchanged). With a path declared, `lock` fingerprints that state before and after the command. Under `--no-pause` with a declaring server still running, a change is a `resource-mutated` failure naming the servers to stop and the command to re-run, because the running server holds the old state open and can write its cached pages back over what the command wrote. Under the default paused mode the same change is just a note.

This closes a silent data loss: a migration run under `--no-pause` that wiped and rebuilt a local database reported success while the seeded rows were gone, and nothing in the output distinguished that from a clean run. The check is deliberately modest about its limits, and the contract states them: it flags the risk window rather than the damage, it cannot see state outside the declared path or divergence that never reaches disk, and above 8 MiB it samples a file rather than hashing it whole.
7 changes: 7 additions & 0 deletions .changeset/lock-parsing-and-contention.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"devctl": minor
---

`devctl lock` no longer passes its own options to the guarded command. `devctl lock d1 --timeout 300 -- cmd` ran `env --timeout 300 -- cmd` and died with `env: illegal option -- t`, because the resource name ended option parsing and everything after it was captured as the command. The command is now taken from after `--` verbatim, so a nested `--`, a dash option, and an empty string all survive, while a missing terminator or an unknown option is rejected instead of quietly passed through.

A contended `devctl lock` says who holds the resource instead of blocking silently for up to five minutes. It names the holder's pid, how long that run has been going, and which servers it paused or left running, then repeats a still-waiting line while it waits. Silence there reads as a hung gate, and the reflex it invites is killing the run that holds the lock, which is the one making progress. `--acquire-timeout 0` now makes exactly one attempt and fails immediately, which it could not do before: the wait loop never ran its body at a zero budget and failed with a message naming no holder. All of `lock`'s own output moved to stderr, so stdout carries only the guarded command's.
2 changes: 1 addition & 1 deletion .changeset/port-claim-settle-why.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
"minor"
"devctl": minor
---

Lock resume waits until claimed ports are free before re-ensuring, so a pause no longer races a dirty bind. Composite servers declare a port span or named subports so sibling worktrees rebind a whole claim block (relative offsets move, absolute ports stay singleton), and `devctl why` keeps the refusal lines from the last run across ensure retries instead of going blank on exit 0.
38 changes: 38 additions & 0 deletions .github/workflows/changesets.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Changesets are .md files, and the CI gate ignores markdown to keep macOS
# minutes down, so nothing read them before they merged. One with malformed
# frontmatter therefore passed every check and then failed the Release workflow
# on main, silently blocking every publish until someone read that log.
#
# The check is a direct parse rather than `changeset status`, which compares
# against a base branch and so needs full history and a local main that a
# pull_request checkout does not have. Shape is the failure class; which
# packages changed is not what this is guarding.
name: Changesets

on:
pull_request:
paths:
- '.changeset/**'
- 'package.json'
- 'scripts/validate-changesets.mjs'

concurrency:
group: changesets-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read

jobs:
validate:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: "24"

- name: Validate changeset frontmatter
run: node scripts/validate-changesets.mjs
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,18 @@ Identity and stack
- Three products, one daemon: devctld owns all server processes; devctl (CLI) and devctl.app (SwiftUI MenuBarExtra) are thin clients over a unix socket (default ~/Library/Application Support/devctl/daemon.sock; DEVCTL_SOCKET overrides; /tmp fallback near the sun_path limit), NDJSON protocol. devctl daemon install/uninstall/start/stop/restart manage the LaunchAgent (dev.quantizor.devctl); tests and the smoke gate run devctld --foreground.

Codebase map
- Sources/DevCtlKit: shared core, the unit-test target of record. Models.swift (specs, phases, ServerStatus), Wire.swift (JSONCoding, typed request/response/event frames, NDJSON framing, stable error codes), Client/DaemonClient.swift (blocking-POSIX socket actor used unchanged by CLI and app), Paths/Paths.swift (path constants, canonical project path, atomic write + defensive load with per-call unique temp names so two writers in one process cannot rename each other's temp away, portable SHA-256; agent.path holds login-shell PATH for the daemon), Setup/SetupPlanner.swift (first-run / upgrade decisions, harness offers, stage-and-rename binary install), Agent/ (AgentContext: the pure session-context renderer the hook injects, bad-state servers first with a devctl why recommendation and devctl's own stderr count, never raw child output; DiscoveryStanza), Net/LoopbackProbe.swift (dual-stack loopback listen probe shared by the daemon port pre-check and CLI doctor), Net/PortClaim.swift + PortMaterializer.swift (effectivePort claim: portSpan and named ports; env injection and URL rewrite), Launchd/ (LaunchdAdmin: dual install path; SMAppService via app deep link when /Applications/devctl.app exists, else legacy home LaunchAgent + Application Support bin/devctld; --legacy forces the home path; DaemonRecoveryPolicy decides whether an unreachable daemon is auto-restarted; AgentRebindPolicy + agent.rebind settle the ad-hoc CDHash window on DMG replace), DeepLink/ (parse/serialize + DeepLinkRunner + notification action map), Log/DevCtlLog.swift (OSLog facade with a recording backend for tests).
- Sources/DevCtlKit: shared core, the unit-test target of record. Models.swift (specs, phases, ServerStatus), Wire.swift (JSONCoding, typed request/response/event frames, NDJSON framing, stable error codes), Client/DaemonClient.swift (blocking-POSIX socket actor used unchanged by CLI and app), Paths/Paths.swift (path constants, canonical project path, atomic write + defensive load with per-call unique temp names so two writers in one process cannot rename each other's temp away, portable SHA-256; agent.path holds login-shell PATH for the daemon), Setup/SetupPlanner.swift (first-run / upgrade decisions, harness offers, stage-and-rename binary install), Agent/ (AgentContext: the pure session-context renderer the hook injects, bad-state servers first with a devctl why recommendation and devctl's own stderr count, never raw child output; DiscoveryStanza), Net/LoopbackProbe.swift (dual-stack loopback listen probe shared by the daemon port pre-check and CLI doctor), Net/PortClaim.swift + PortMaterializer.swift (effectivePort claim: portSpan and named ports; env injection and URL rewrite, including resolving a root-relative head or healthcheck url against the server's own base), Config/ (ProjectConfig loader and validator; ConfigProjection projects merged specs back down to devservers.json, dropping everything the machine derived, for `config init`; EffectiveHost is the one home for the host a spawn will use, read by both prepareSpawn and config check; LocalOverlay; LockResource reads locks declarations and resolves a resource's state path), Resource/ResourceIdentity.swift (bounded fingerprint of a lock resource's state so `lock` can report a change made under a live holder), Launchd/ (LaunchdAdmin: dual install path; SMAppService via app deep link when /Applications/devctl.app exists, else legacy home LaunchAgent + Application Support bin/devctld; --legacy forces the home path; DaemonRecoveryPolicy decides whether an unreachable daemon is auto-restarted; AgentRebindPolicy + agent.rebind settle the ad-hoc CDHash window on DMG replace), DeepLink/ (parse/serialize + DeepLinkRunner + notification action map), Log/DevCtlLog.swift (OSLog facade with a recording backend for tests).
- Sources/DevCtlDaemonCore: daemon logic as a library. Supervisor/ (ServerSupervisor actor per server: spawn, spool capture, health-gated phase machine, ensure/wait, group + descendant teardown with ProcessIdentity start-time revalidation; the ProcessLauncher seam; ProcessTree QA1123 sysctl sweep), Health/HealthProber.swift (EffectiveHealthcheck resolution, the HealthProber seam with ephemeral URLSession HTTP probes + BSD TCP, and PortGuard's lsof diagnostics, which live in that same file), Registry/ (owner of registry.json and state.json), Control/ (Router method dispatch + port pre-check + persisted resource locks with daemon-owned pause/resume and dead-holder auto-release + NWListener ControlServer with stateUpdateHandlers).
- Sources/devctld: thin main; identical behavior under launchd and --foreground (tests and the smoke gate use foreground). Applies agent.path into process env before spawn.
- Tests: DevCtlKitTests is the unit-test target of record and holds the schema goldens; DevCtlDaemonCoreTests drives a real Router over temp paths; DevCtlCLITests covers CLI behavior with a contract and no other way to exercise it (argument parsing, the lock notices and identity verdict), importing the executable target with @testable.
- Sources/devctl: CLI (swift-argument-parser). Two files only: HookSupport.swift (HookContext, the thin socket fetch over DevCtlKit's AgentContext renderer, + HarnessAdapter registry; adding a harness: CONTRIBUTING.md) and CLI.swift, which holds every command as a struct, including Switch (branch switching + lifecycle playbooks), Lock (run-under-resource-lock), Doctor (health report; owns the cross-project port-collision and squatter findings), and Link / x-url (deep links). CLI.swift is past the size where splitting is worth asking about.
- Sources/DevCtlApp: menu bar app (DaemonModel 2s-polling model + crash notifications with Open/Why actions; AgentService wraps SMAppService.agent for Login Items registration, escalating to unregister + register when the agent reads `enabled` but the socket stays silent (a bootout or replaced bundle leaves the registration intact with nothing loaded) and reporting `requiresApproval` as its own failure instead of retrying; unregister records the stop intent so the recovery poll does not undo it; an unreachable daemon self-recovers via AgentService under DaemonRecoveryPolicy, falling back to legacy LaunchdAdmin only when the bundle carries no agent plist; DaemonDownRow offers Start, or Open Login Items while approval is pending, since a deliberate `daemon stop` stands down auto recovery; PresenceLabel is AppKit-drawn colored tally dots only with renderingMode(.original); popover autogrows to a cap; nested head rows with UserDefaults-persisted pins; DashboardView logs/timeline/config tabs; SpotlightIndexer named Core Spotlight index; AppDeepLink handles `devctl://` including `daemon/ensure` and `daemon/unregister`; SetupPanel first-run / upgrade installer from bundle Resources). Pure DaemonClient consumer.
- Sources/fixture-server: test double dev server (heartbeat printer; TCP-listen, timed-exit, grandchild, ignore-sigterm, binary, flood modes; see its header comment).

Commands
- make build: swift build -c release (all products)
- make test: swift test; budget under 30s, the run prints the live timing
- scripts/smoke.sh: the end-to-end gate. Debug-builds, boots a real devctld on a temp socket, then asserts register/start/status, spool capture, health/ensure/wait, port conflicts, marks/events/why, resource locks (pause + refused ensure + resume), whole-group death on stop, child survival across a daemon kill, `link`/`x-url` deep-link dispatch, and that the assembled app declares `CFBundleURLSchemes=devctl`, ships AppIcon + CLI + daemon in Resources, and ships Helpers/devctld plus the in-bundle LaunchAgents BundleProgram plist. Run it after touching the supervisor, wire protocol, CLI, or deep links.
- scripts/smoke.sh: the end-to-end gate. Debug-builds, boots a real devctld on a temp socket, then asserts register/start/status, spool capture, health/ensure/wait, port conflicts, marks/events/why, config recovery (`config init` round-tripping through `config check`, its refusal to clobber, `register --write`), relative-head resolution and its `config check` rejection, resource locks (pause + refused ensure + resume, option parsing, a contended acquire naming the holder, and the identity check firing under `--no-pause` while staying silent otherwise), whole-group death on stop, child survival across a daemon kill, `link`/`x-url` deep-link dispatch, and that the assembled app declares `CFBundleURLSchemes=devctl`, ships AppIcon + CLI + daemon in Resources, and ships Helpers/devctld plus the in-bundle LaunchAgents BundleProgram plist. Run it after touching the supervisor, wire protocol, CLI, or deep links.
- scripts/smoke-deeplink.sh: Launch Services E2E for `devctl://` (warm + cold `open`). Requires a GUI session; run before merging URL-scheme work. OSLog scrape is strict on a tty (`DEVCTL_OSLOG_STRICT=1` forces it).
- scripts/smoke-launchd.sh: the REAL LaunchAgent lifecycle via `daemon install --legacy` (install, restart bounce + re-ensure, install-upgrade bounce + re-ensure, deliberate-stop intent, auto-bootstrap resurrection, uninstall). Mutates the user launchd domain; refuses to run if a home plist or bootstrapped job already exists; leaves nothing behind. SMAppService is exercised by installing from the app on a GUI session.
- make app: assembles fat devctl.app via scripts/make-app-bundle.sh (CLI in Contents/Resources; signed Helpers/devctld + Contents/Library/LaunchAgents BundleProgram plist for SMAppService; AppIcon.icns; ad-hoc signed; SIGN_IDENTITY upgrades; declares the `devctl://` URL scheme). make dmg: UDZO image via scripts/make-dmg.sh, holding the app alone (no /Applications symlink: the app installs itself after an in-app confirm) over a background rendered by scripts/make-dmg-background.swift that says to double-click. Finder window layout needs a GUI session and a volume name that is not already mounted; on a headless runner that pass is skipped and the image still ships. scripts/notarize.sh: notarytool + staple. make install: CLI + daemon to ~/.local/bin, app to /Applications, daemon install.
Expand Down
28 changes: 2 additions & 26 deletions BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,31 +14,7 @@ Open work only; entries are removed by the change that resolves them.
- `IntegrationTests` is a single `placeholder()` while `docs/design.md` promises a real end-to-end suite there (port-conflict from a second project, concurrent double-ensure). Those now live in `scripts/smoke.sh` and unit suites; either build the integration target out or retire the promise in the design doc.
- A restarting daemon is indistinguishable from a dead one for the length of `recoverAtStartup`, which is seconds when there is real state to restore. The listener only reaches `.ready` after restore, and the socket is unlinked in `ControlServer.init`, so clients get `daemon-unreachable` with ENOENT and nothing that says "starting". Moving the unlink later only changes the errno, since the daemon is unreachable either way: the fix is to answer during restore (bind early and reply "starting" to everything, or write a state file clients can read) so `devctl daemon status` can say restoring rather than down. Wanted because install/restart bounce servers, and an agent polling through that window sees a daemon that looks gone.
- Split `CLI.swift`: every command struct lives in that one file, which is past the size where splitting is worth asking about. A structure decision, not drift; the codebase map describes the current layout.
- `lock` mis-parses its own options after the resource: `devctl lock d1 --timeout 300 -- <cmd>` passed `--timeout` through to the command, which failed with `env: illegal option -- t`. The same call without `--timeout` worked. Either the option has to precede the resource (undocumented, and `--help` lists it after `<resource>`) or the parser is leaking pre-`--` options into the command vector. A stray flag reaching the command is a silent behavior change in the worst case, not just a usage error.
- Field-level config editing in the dashboard would preserve `devservers.json` formatting instead of normalizing writes. `devctl config init` writes indented JSON with sorted keys, so a recovered file and a dashboard-saved file now agree on shape, but a hand-authored one with its own ordering or comments still normalizes on save.

## `lock --no-pause` is not enough when the command deletes the locked state

A session wiped a project's local database directory to re-run migrations from
scratch, under `devctl lock d1 --no-pause`. The lock serialized access, but the
dev server kept the file open across the deletion and flushed its cached pages
back over the freshly migrated file. The migration reported success, the ledger
recorded every file as applied, and the seeded rows were gone. Three separate
wrong diagnoses followed before reading the sqlite file's raw bytes settled it,
and one of those wrong diagnoses got as far as a new guardrail before being
disproved.

`--no-pause` is documented as "the server tolerates staying up", which reads as a
property of the *server*. The property that actually matters is a property of the
*command*: whether it mutates the resource in place (fine) or removes and
recreates it (not fine, the open handle wins).

Worth considering:

- Refuse `--no-pause` when the command line touches the locked resource's own
state path with a removing verb (`rm`, `mv`, `rmdir`), or at least warn.
- Or make `lock` report, on completion, that the resource's backing file changed
identity (inode/hash) while a holder was up, which is the observable tell.

Either turns a silent data loss into a loud refusal. Right now nothing in the
output distinguishes "migrated and seeded" from "migrated, seeded, and clobbered".
- Exact hashing above the file cap needs a streaming SHA-256. `SHA256Portable.digest` takes a whole `[UInt8]` with no incremental entry point, so a lock resource larger than 8 MiB is fingerprinted by head, tail, size, and mtime, and a middle-only rewrite that preserves all four escapes the identity check. The limit is asserted in `ResourceIdentityTests` and stated in the contract rather than left implicit.
- The identity check flags the risk window, not the damage: the flush that corrupts can land after the guarded command exits and the second capture is taken. Catching that would need the daemon to watch the resource across the whole hold, or to compare again once the paused set has resumed.
11 changes: 11 additions & 0 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,17 @@ let package = Package(
name: "DevCtlDaemonCoreTests",
dependencies: ["DevCtlDaemonCore", "DevCtlKit"]
),
/** The CLI's argument parsing is behavior with a contract (docs/cli-contract.md)
and no other way to exercise it: a parse defect there silently changes
what a guarded command receives. */
.testTarget(
name: "DevCtlCLITests",
dependencies: [
"DevCtlKit",
"devctl",
.product(name: "ArgumentParser", package: "swift-argument-parser"),
]
),
.testTarget(
name: "IntegrationTests",
dependencies: ["DevCtlKit"]
Expand Down
Loading
Loading