Skip to content

Add the formal-verification specify skill's propose, create and check forms #15773

Description

@kyle-sexton

Decision (owner, 2026-11-12)

Slice 2 of 3 of the formal-verification plugin. The whole design, the directory layout, the decided names and the design rules are on slice 1: Add the formal-verification plugin's chart skill, setup and consumer contract, proven on the loop-lane claim protocol.

Source: her create-formal-verification-skill (cursor/plugins pstack/skills/, MIT), text kept as far as possible.

Depends on

This item

/formal-verification:specify with its bare, create and check forms.

  • Bare: propose the promise worth proving (promise, harm, tool, and the folder create would write); write nothing.
  • create: pick the target from the code ("A good target is a promise the code makes, where a break harms users and tests rarely reach the case that breaks it. Choose the strongest such target in the repo, not the one that is easiest to verify"); let the question choose the tool (a model checker for bad orderings, a code-level verifier for every-input properties, a proof when a bound is not enough, generated-input tests through /testing:write when no verifier exists for the language); generate the model and the consumer's formal-<area> skill with its sync.yaml; then break the code on a scratch change and confirm the check catches it before hand-off.
    • create takes a promise as its argument: /formal-verification:specify create "<promise>".
    • Tool choice: with no tool recorded, create researches and recommends one, asks, and records the choice only in that area's formal-<area> skill ("Tool and why"). A later create reads the existing formal-* skills first.
    • specify never installs a verification tool. For a tool setup does not declare, it names the tool, the reason and the tool's official install page, and stops; unattended it reports "blocked: tool missing".
    • Attended, with no target named, create shows the bare form's proposal (promise, harm, tool, folder) and waits for a yes before writing. create --unattended is a dry run that writes nothing and lists its open questions.
    • The seeded break is made in a throwaway git worktree, never the person's working tree, and recorded in the formal-<area> skill. A reused recorded break is still followed by a full rerun. The discarded worktree is outside the unattended write boundary.
    • When no verifier runs on the language, create hands off to /testing:write's property guidance, writes no formal-<area> skill, and reports that the area has property tests, not formal verification.
  • check: a deterministic script and a pass or fail gate on whether a consumer's formal-<area> skill meets the contract. It edits nothing. It fails on: missing or wrong frontmatter, a missing required section, a missing or invalid sync.yaml, a sync commit that is not an ancestor of HEAD, a covered path missing at HEAD, no run command. It warns on covered code changed since the sync commit and on verification skills outside the contract. A per-check setting on_stale (warn or fail, default warn) can make stale coverage fail.
  • This repository runs check as a step inside CI Status over .claude/skills/formal-* once a formal-<area> skill exists; it adds no new required check to a ruleset.
  • Her quality bar is kept: true to the code, able to fail, findings stay findings (never weaken a property, never change product code unless asked), honest about scope, readable without the tool, cheap to rerun, kept in step with the code.
  • specify names no tool. Her TLA+ writing rules stay in chart; specify gets no TLA+ reference file, and keeps a tool-neutral choose-a-tool.md. Her sentence "TLA+ is the default" is replaced, recorded as a change (ADR 0101).
  • specify is model-invocable (disable-model-invocation: false), because /corrections:codify chains to it.
  • This slice does not wait for Route codify findings by capability and hand the build to implement. The enforcement ladder's formal-verification rung is opt-in, so this slice adds the line to this repository's AGENTS.md: "This repository opts in to formal verification: /corrections:codify may place a rule on the enforcement ladder's formal-verification rung."

Before build

The interview ran and the owner confirmed it on 2026-11-14 ("Go with recommendations, whatever the research says, and whatever is the best fit in our repo", then "Confirmed."). The contract's required sections are decided on #15648; what check treats as a failure and how a model invokes the skill are stated above.

Left to planning: how the dogfood's /app-control:drive run is wired (the control skill's feature row, what happens if the drive does not resolve, how many candidates it tries).

Dogfood

A second promise in this repository, chosen by the skill's own bare run, taken through create with the seeded-break proof. The bare form's proposal is taken only if it is a promise other than the claim protocol; if its tool does not resolve or is not installed, the run records that and takes the next candidate.

Owner step

The dogfood's attended create waits for a person's yes on the bare form's proposal before it writes.

Done when

  • Bare /formal-verification:specify proposes a promise, its harm, its tool and the folder create would write, and changes no file.
  • /formal-verification:specify create "<promise>" takes a promise as its argument.
  • /formal-verification:specify skill and reference text name no verification tool, and specify has no TLA+ reference file.
  • /formal-verification:specify no longer carries her sentence "TLA+ is the default", and the change is recorded with ADR 0101 as its reason.
  • With no tool recorded for an area, /formal-verification:specify create researches and recommends a tool, asks, and records the choice only in that area's formal-<area> skill under "Tool and why".
  • /formal-verification:specify create reads the existing formal-* skills before it chooses a tool.
  • /formal-verification:specify installs no verification tool.
  • For a tool setup does not declare, /formal-verification:specify names the tool, the reason and the tool's official install page, and stops.
  • Unattended, with its tool missing, /formal-verification:specify reports "blocked: tool missing".
  • Attended /formal-verification:specify create with no target named shows the bare form's proposal and writes nothing until a yes.
  • /formal-verification:specify create --unattended writes nothing and lists its open questions.
  • /formal-verification:specify create makes its seeded break in a throwaway git worktree, never the person's working tree, and the generated check catches it before hand-off.
  • The formal-<area> skill /formal-verification:specify create generates records its seeded break and meets the formal-verification consumer contract.
  • When /formal-verification:specify reuses a recorded seeded break, a full rerun of the check follows.
  • When no verifier runs on the language, /formal-verification:specify create hands off to /testing:write, writes no formal-<area> skill, and reports property tests, not formal verification.
  • /formal-verification:specify keeps her quality bar: true to the code, able to fail, findings stay findings, honest about scope, readable without the tool, cheap to rerun, kept in step with the code.
  • /formal-verification:specify reaches other skills only by slash reference with an "if installed" fallback and copies none of chart's commands.
  • /formal-verification:specify is model-invocable (disable-model-invocation: false).
  • The formal-verification check script is deterministic and edits no file.
  • The formal-verification check script exits non-zero on missing or wrong frontmatter, a missing required section, a missing or invalid sync.yaml, or no run command.
  • The formal-verification check script exits non-zero on a sync commit that is not an ancestor of HEAD and on a covered path missing at HEAD.
  • The formal-verification check script warns on covered code changed since the sync commit and on verification skills outside the contract.
  • The formal-verification check script reads a per-check on_stale setting (warn or fail, default warn), and fail makes stale coverage fail.
  • The formal-verification check script runs as a step inside CI Status over .claude/skills/formal-* once a formal-<area> skill exists.
  • This repository's AGENTS.md carries the line "This repository opts in to formal verification: /corrections:codify may place a rule on the enforcement ladder's formal-verification rung."
  • The formal-verification dogfood takes a promise other than the claim protocol from the bare form's proposal through create, and its seeded break is caught.
  • If the formal-verification dogfood's proposed tool does not resolve or is not installed, the run records that and takes the next candidate.

Out of scope

Activity

  1. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Nov 13, 2026
  2. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Large
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none

    I read this as a new capability: the formal-verification plugin's specify skill with bare, create and check forms. It also covers the deterministic check script, a CI Status step, an opt-in line in AGENTS.md and a dogfood run.

    The newest decision comment is the owner's decision recorded 2026-11-17. It settles the one open question: an undeclared tool stops create only when that tool is missing. The earlier decision comment settles the rest. No open intent questions remain, and the dogfood's attended yes is a person step the lane cannot run.

    #15648 is not in the open list, so I treat it as done and list no blocker. I found no merged pull request that already covers this. The work is large and touches protected paths, so it likely needs splitting. The issue's own note says a host session is already building it.

    Automated triage. A maintainer reviews it.

  3. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    Contributor

    The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

  4. kyle-sexton commented on Nov 14, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. Interview complete, owner-confirmed 2026-11-14 ("Go with recommendations, whatever the research says, and whatever is the best fit in our repo", then "Confirmed."). The shared decisions are on issue #15648. Decisions for this slice:

    • Her TLA+ writing rules stay in chart; specify gets no reference/tools/tla-plus.md (correction to the body, per ADR 0101).
    • Tool choice: with no tool recorded, create researches and recommends one, asks, and records the choice only in that area's formal-<area> skill ("Tool and why"). A later create reads the existing formal-* skills first.
    • specify never installs a verification tool. For a tool setup does not declare, it names the tool, the reason and the tool's official install page, and stops; unattended it reports "blocked: tool missing".
    • Attended, with no target named, create shows the bare form's proposal (promise, harm, tool, folder) and waits for a yes before writing. create --unattended is a dry run that writes nothing and lists its open questions.
    • The seeded break is made in a throwaway git worktree, never the person's working tree, and recorded in the formal-<area> skill. A reused recorded break is still followed by a full rerun. The discarded worktree is outside the unattended write boundary.
    • check is a deterministic script. It fails on: missing or wrong frontmatter, a missing required section, a missing or invalid sync.yaml, a sync commit that is not an ancestor of HEAD, a covered path missing at HEAD, no run command. It warns on covered code changed since the sync commit and on verification skills outside the contract. A per-check setting on_stale (warn or fail, default warn) can make stale coverage fail.
    • This repository runs check in CI over .claude/skills/formal-*, required once a formal-<area> skill exists.
    • When no verifier runs on the language, create hands off to /testing:write's property guidance, writes no formal-<area> skill, and reports that the area has property tests, not formal verification.
    • This slice does not wait for Release enforcement ladder 3.0.0 with the verification tier and capability-named rungs #15708 or Route codify findings by capability and hand the build to implement #15728: it takes a promise as its argument (/formal-verification:specify create "<promise>"). The enforcement ladder's formal-verification rung is opt-in, so this slice adds the line to this repository's AGENTS.md: "This repository opts in to formal verification: /corrections:codify may place a rule on the enforcement ladder's formal-verification rung."
    • specify is model-invocable (disable-model-invocation: false), because /corrections:codify chains to it.
    • Dogfood: the bare form's proposal is taken only if it is a promise other than the claim protocol; if its tool does not resolve or is not installed, the run records that and takes the next candidate.

    Left to planning: how the dogfood's /app-control:drive run is wired (the control skill's feature row, what happens if the drive does not resolve, how many candidates it tries).

  5. removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Nov 17, 2026
  6. github-actions commented on Nov 17, 2026

    @github-actions
    Contributor
    No description provided.
  7. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 17, 2026
  8. melodic-automation-lanes commented on Nov 17, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue lane stopped on this issue (patch-protected-path): the change needs a path the issue lane may not push, which a maintainer session builds by hand.
    It added host-run and set Work status to confirmed: an agent run on a host builds it, and it waits on no decision from a person.

    In the run's words:

    The run stopped early (patch-protected-path): its change needs a path the issue lane may not push, which a person changes instead. In the run's words: patch-protected-path: .github/workflows/pr-require-checks.yml (the done-when item 'The formal-verification check script runs as a step inside CI Status over .claude/skills/formal-*' needs a new step in the ci-status job defined in this workflow, and a formal-<area> skill, .claude/skills/formal-claim-protocol/, already exists, so the step is due now) and .claude/skills/formal-<area>/ (the dogfood done-when item writes a second consumer formal-<area> skill under .claude/skills/, a Claude Code protected path). The lane cannot push either path; the specify skill, check script and AGENTS.md line could land here, but the issue's done-when needs both protected writes, and the dogfood also waits on a person's attended yes (Owner step).

    What the run says it did not do:

    • The whole issue: /formal-verification:specify (bare, create, check), the check script, the CI Status step, the AGENTS.md opt-in line and the dogfood. Nothing was built because the issue needs protected-path writes (.github/workflows/pr-require-checks.yml, .claude/skills/formal-<area>/) and an…
    • Repository checks (check-changelog-fragments, check-changed-skills, check-affected) were not run: the run stopped before any change, so there is no diff for them to check.

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/43996584540

    To hand it back to the lane, remove host-run and set Work status to agent-ready again.

  9. self-assigned this
    on Nov 17, 2026
  10. kyle-sexton commented on Nov 17, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. Owner, 2026-11-17, in session ("Go with recommendations."), posted by an agent session: for a tool setup does not declare, /formal-verification:specify names the tool, the reason and its official install page and stops when that tool is missing. An undeclared tool that is already installed and runs does not stop create. Read literally, create could only ever finish with the one tool setup declares, which contradicts letting the question choose the tool.

    Status: a host session is building this issue after the lane's patch-protected-path stop (branch feat/formal-verification-specify, no pull request yet). The dogfood follows in a second pull request after the owner's yes on the bare form's proposal.

  11. melodic-automation-lanes commented on Nov 17, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.

Fields

Work status

confirmed

Priority

medium

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions