Skip to content

Harden devctl lock, and unblock releases - #13

Merged
quantizor merged 9 commits into
mainfrom
fix/lock-hardening
Aug 8, 2026
Merged

Harden devctl lock, and unblock releases#13
quantizor merged 9 commits into
mainfrom
fix/lock-hardening

Conversation

@quantizor

@quantizor quantizor commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Releases have been silently failing since #10, and devctl lock had three ways to mislead you. Both are fixed here.

Releases publish again. A changeset with malformed frontmatter could not be parsed, so the Release workflow failed on every push to main and nothing shipped. Three merged pull requests are still waiting on it. Changesets are markdown, and the build gate skips markdown to save CI minutes, so nothing read one before it merged. A new check parses changeset frontmatter on any pull request that touches it.

devctl lock accepts its own options. Passing --timeout or --acquire-timeout before the -- used to hand them to the guarded command, which then failed with a confusing error from env. Options now reach the lock and the command receives exactly what follows --, including a nested -- or an empty argument. Forgetting the -- entirely, or misspelling an option, is now rejected instead of silently changing what runs.

A contended lock says who holds it. It used to block in silence, which reads as a hang and invites killing the run that is actually making progress. It now names the holder, how long that run has been going, and which servers it paused or left up, and repeats while it waits. --acquire-timeout 0 fails immediately for scripts that would rather not wait. All of this goes to stderr, so the guarded command still owns stdout.

--no-pause no longer accepts a command that destroys the state it guards. A migration run that wiped and rebuilt a local database reported success while the seeded rows were gone: the lock serialized access, but the still-running server held the old files open and wrote its cached pages back over them. A locks entry can now say where its state lives, and lock checks that state around the command. A change made while a declaring server stayed up is a loud failure naming the servers to stop and the command to re-run; the same change with nothing running is just a note. Locks written as plain names keep working exactly as before.

The check is deliberately modest, and the CLI contract says where it stops: it flags the risk rather than proving damage, sees only the path you declare, and samples very large files rather than reading them whole.

Docs move with the code: the codebase map covers the new modules and test target, and the design record no longer claims register --write never shipped.

Verified with the unit suite and the end-to-end smoke gate, the latter repeated to confirm the new timing assertions are stable.

… holder

The parse defect was structural. `.captureForPassthrough` ends the option loop
at the first positional value, so the resource itself stopped option parsing and
`--timeout 300 -- cmd` all landed in the command vector. Declaration order can
never fix that, and the comment claiming it could was wrong, as was the one
blaming @Flag inversion for the same symptom. `.postTerminator` lifts everything
after `--` before positionals are filled, which also deletes the hand-scrape
that rescued only `--no-pause` and makes --help finally show the terminator the
contract documents.

The acquire wait was silent, so a second run looked hung and the reflex was to
kill whichever run held the lock. A new read-only lock.status answers who holds
it, and the CLI names the pid, the hold's age and what it paused or left
running, then repeats while waiting. The loop became a schedule so
--acquire-timeout 0 makes one attempt; the old `while Date() < deadline` never
entered its body at budget 0 and failed naming no holder. Every notice moved to
stderr, which lets the smoke gate parse lock's stdout as plain JSON.

New CLI test target: the parse behavior has a contract and no other way to
exercise it. Its tests fail on the old strategy with the reported argv.
… holder

The incident this closes: a session wiped a local database directory to re-run
migrations under --no-pause. The lock serialized access, the still-running
server held the old file open and flushed its cached pages back over the
migrated one, and the migration reported success while the seeded rows were
gone. Three wrong diagnoses followed before the raw bytes settled it, because
nothing in the output distinguished that run from a clean one.

A locks entry may now name its state path, as an object beside the bare string
form, which keeps parsing and re-encoding unchanged; that compatibility is the
headline test. lock fingerprints the state around the guarded command and
reports a change: a fault under --no-pause with a live declarer, a note
otherwise. Inode is in the fingerprint because the incident's shape is a
delete-and-recreate that a content hash calls identical.

Two declarers naming different paths for one resource refuses rather than
guessing which state a lock guards. The directory walk sorts before clipping so
a truncated manifest stays deterministic, and it keeps each entry's own absolute
path: rebuilding one from a stripped relative produced paths that existed
nowhere, silently stat-failed, and left every capture comparing equal, which the
entry-count assertion caught.

What the check cannot catch is stated in the contract rather than implied.
The codebase map gained no entry for Config/ConfigProjection, EffectiveHost,
LockResource, Resource/ResourceIdentity, or the new CLI test target, and the
smoke gate's description listed none of the assertions added with them. The
design record still said `register --write` did not ship and that nothing writes
devservers.json back, and its config model documented neither `locks` nor the
two shapes a head may take. README now says how a lost gitignored config is
recovered instead of telling the reader to keep a copy off the machine.
Copilot AI lite review requested due to automatic review settings August 8, 2026 03:50
#10

.changeset/port-claim-settle-why.md declared `"minor"` with no package name, so
Changesets refused to parse it and the Release workflow has failed on every push
to main since 2026-07-28. Nothing was released in that window: #10, #11 and #12
all merged with their changesets stranded.

It reached main because the CI gate ignores markdown to keep macOS minutes down,
and a changeset is markdown, so no check ever read one before it merged. A
separate Linux job now runs `changeset status` on any pull request touching
.changeset, which fails in seconds on exactly this frontmatter and leaves the
expensive macOS gate's path filter alone. Verified both ways: exit 1 with the
same error CI reported, exit 0 once the frontmatter names the package.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens devctl lock end-to-end (CLI parsing, contention UX, and a guard against silent state clobbering) by updating the CLI, wire protocol, daemon lock behavior, config schema, and tests to match the documented contract.

Changes:

  • Fix devctl lock argument parsing by requiring -- and capturing the guarded command verbatim via .postTerminator, rejecting missing terminators and unknown options.
  • Add contention visibility via lock.status so waiting acquires can name the lock holder (pid, age, paused/live servers) and provide bounded, repeated stderr notices.
  • Add optional lock state-path declarations and resource fingerprinting to detect and report mutations, escalating to a new resource-mutated error under --no-pause with live declarers.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
Tests/DevCtlKitTests/ResourceIdentityTests.swift New unit tests for ResourceFingerprint behaviors (inode changes, sampling caps, directory determinism expectations).
Tests/DevCtlKitTests/ProjectConfigTests.swift Tests for bare-string vs object-form lock declarations and state-path resolution/conflict behavior.
Tests/DevCtlDaemonCoreTests/ResourceLockTests.swift Router-level tests for lock.status and --no-pause holder recording.
Tests/DevCtlCLITests/LockWaitTests.swift New CLI-unit tests for contention notice wording and acquire scheduling behavior.
Tests/DevCtlCLITests/LockParsingTests.swift New CLI-unit tests asserting the corrected terminator-based parsing contract.
Tests/DevCtlCLITests/LockIdentityTests.swift New CLI-unit tests for identity-verdict messaging and hint composition.
Sources/DevCtlKit/Resource/ResourceIdentity.swift Implements resource fingerprint capture/compare logic (files, directories, symlinks) with caps/sampling.
Sources/DevCtlKit/Protocol/Wire.swift Adds resource-mutated code, lock.status method, and expands lock holder/result payloads.
Sources/DevCtlKit/Paths/Paths.swift Refactors SHA256 helpers to a full-hex function and redefines hash8 as its prefix.
Sources/DevCtlKit/Model/Models.swift Introduces LockDeclaration (bare string or {name,path}) and updates ServerSpec.locks type.
Sources/DevCtlKit/Config/ProjectConfig.swift Updates project config model to use LockDeclaration for locks.
Sources/DevCtlKit/Config/LockResource.swift New helper to centralize declarer detection and state-path resolution across specs.
Sources/DevCtlDaemonCore/Supervisor/ServerSupervisor.swift Adapts server status locks reporting to the new declaration type (names only).
Sources/DevCtlDaemonCore/Control/ControlServer.swift Adds lock.status routing and records lock holder pause/live metadata plus resolved statePath.
Sources/devctl/CLI.swift Implements new parsing strategy, contention notices to stderr, acquire scheduling, and identity guard behavior.
scripts/smoke.sh Extends smoke coverage for option leakage, contention visibility, fail-fast acquire, and identity checking.
README.md Updates CLI feature list and clarifies recovery via devctl config init.
Package.swift Adds DevCtlCLITests test target for CLI parsing/contract behavior.
docs/design.md Updates references to AGENTS.md and documents locks object form and identity checking behavior.
docs/cli-contract.md Documents new resource-mutated code and the updated devctl lock contract details.
BACKLOG.md Removes resolved lock parsing/no-pause clobber items; adds follow-ups for streaming hashing and post-hold risk window.
AGENTS.md Updates codebase map and smoke coverage description to include new lock/config behaviors.
.changeset/lock-parsing-and-contention.md Changeset for lock parsing fix and contention notices.
.changeset/lock-identity-guard.md Changeset for identity guard and new failure mode under --no-pause.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Sources/DevCtlDaemonCore/Control/ControlServer.swift
Comment thread Sources/devctl/CLI.swift
Comment thread Sources/devctl/CLI.swift Outdated
Comment thread Sources/devctl/CLI.swift Outdated
Comment thread Sources/DevCtlKit/Resource/ResourceIdentity.swift Outdated
npm ci resolved a different tree on node 22 and failed the lockfile sync check,
so the validator could not run at all. Node 24 is what the Release workflow
uses, and a guard that resolves different dependencies is not checking what
ships.
changeset status diverges from the base branch, and checkout's default shallow
clone has no main to diverge from.
changeset status compares against a base branch, so it needs full history and a
local main that a pull_request checkout does not have, and it pulled a whole npm
install along for a check about file shape. A small script parses the
frontmatter instead: no dependencies, no git, and verified against the exact
form that broke main plus a wrong package name and a bad bump.
@quantizor quantizor changed the title Harden devctl lock: parsing, contention, and a guard against silent clobbering Harden devctl lock, and unblock releases Aug 8, 2026
…es honest

The directory walk stopped early past a soft cap, and FileManager's enumeration
order is unspecified, so an over-cap tree could pick a different subset run to
run and compare unequal to itself. The comment above it claimed sorting before
clipping made the manifest deterministic; the early break made that impossible.
The walk now collects then sorts then clips, which is what the comment says, and
a test captures an over-cap tree twice.

A same-holder re-acquire returned only the paused set, so a client retrying
after a blip lost the live set and the state path and could not judge the
resource for the rest of the hold. The contended notice claimed a --no-pause
holder left servers running even when none were. The resource-mutated message
used a singular subject over a comma-joined list. The acquire loop asked the
daemon who held the lock on every retry, spending hundreds of round trips over
a long wait to print nothing; it now asks only when it is about to speak.

Findings from the Copilot review of #13.
@quantizor
quantizor requested a lite review from Copilot August 8, 2026 04:09
@quantizor

Copy link
Copy Markdown
Owner Author

@pullfrog review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Suppressed comments (4)

Sources/DevCtlDaemonCore/Control/ControlServer.swift:1086

  • acquireLock uses try? await mergedSpecs(...) and then falls back to an empty spec list. When devservers.json is invalid, this can incorrectly acquire the lock without pausing any declarers (and also skip state-path conflict detection), which defeats the lock’s safety guarantees.
        let shouldPause = params.pause ?? true
        let merged = try? await mergedSpecs(project: params.project)
        for spec in merged?.specs ?? []
        where LockResource.declares(resource: params.resource, spec: spec) {

Sources/DevCtlDaemonCore/Control/ControlServer.swift:1113

  • statePath resolution during lock acquire also falls back to merged?.specs ?? []. If config loading failed, this returns nil instead of surfacing config-invalid or conflicting lock-path errors, so the CLI won’t be able to perform the identity check reliably.
        /** Refusing here is correct when devctl cannot tell which state the lock
            guards: taking it anyway would report on the wrong file. */
        let statePath = try LockResource.statePath(
            project: params.project, resource: params.resource, specs: merged?.specs ?? [])

Sources/DevCtlDaemonCore/Control/ControlServer.swift:1079

  • acquireLock swallows mergedSpecs failures (try?) when the same holder re-acquires. If devservers.json is temporarily invalid, this will silently drop statePath (and potentially change behavior mid-hold) instead of returning the config-invalid error the rest of the daemon uses.

This issue also appears in the following locations of the same file:

  • line 1083
  • line 1110
        if let existing = resourceLocks[key], existing.pid == params.holderPid {
            let specs = (try? await mergedSpecs(project: params.project))?.specs ?? []
            return LockResult(
                live: existing.live, paused: existing.paused,
                statePath: try LockResource.statePath(
                    project: params.project, resource: params.resource, specs: specs))

Sources/devctl/CLI.swift:1805

  • On contention, announcedAt is only set when lock.status returns a holder. If the daemon is older (or lock.status fails), announcedAt stays nil, first stays true, and the CLI will attempt lock.status every retry (once per second) despite the comment saying it avoids hundreds of round trips.
                if due {
                    /** An older daemon without lock.status degrades to silence
                        here rather than failing the acquire. */
                    let holder = try? await client.request(
                        .lockStatus,
                        params: LockStatusParams(project: project, resource: resource),
                        expecting: LockStatusResult.self
                    ).holder
                    if let holder {
                        Self.note(
                            first
                                ? LockNotice.contended(
                                    budgetSeconds: acquireTimeout, holder: holder, now: Date(),
                                    resource: resource)
                                : LockNotice.stillWaiting(
                                    elapsedSeconds: elapsed, holder: holder,
                                    remainingSeconds: max(acquireTimeout - elapsed, 0),
                                    resource: resource))
                        announcedAt = elapsed
                    }

Comment thread Sources/DevCtlKit/Config/LockResource.swift
statePath documented its path as project-relative but did not enforce it, so a
committed `{"path": "../.."}` would have pointed the fingerprint at somewhere
the project has no business reading. It now refuses with config-invalid, and a
path that merely contains `..` while staying inside still resolves.

Also repairs a flaky teardown assertion that has failed intermittently since
before this branch: it slept a fixed slice waiting for the descendant sweep to
reap a grandchild, which expires under load and fails a teardown that works. It
polls for the outcome now.

Finding from the Copilot review of #13.
@quantizor

Copy link
Copy Markdown
Owner Author

@pullfrog review

@quantizor
quantizor merged commit 5f24f46 into main Aug 8, 2026
4 checks passed
@quantizor
quantizor deleted the fix/lock-hardening branch August 8, 2026 04:54
@github-actions github-actions Bot mentioned this pull request Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants