Skip to content

playbook.md

Latest commit

 

History

History
1420 lines (1183 loc) · 71.1 KB

playbook.md

File metadata and controls

1420 lines (1183 loc) · 71.1 KB
Raw

Playbook

How to do the recurring tasks in this repository, one procedure per section. AGENTS.md holds the authoring rules, docs/spec.md the composition model; commands use py, and CONTRIBUTING.md, "Setup", gives the launcher on macOS and Linux.

Task Section
Add a base or layer template Add a new base or layer template
Review, rewrite or cut a template Review a template
Turn a downstream lesson into a rule Drain a downstream lesson into a template
Replace a template file Replace a template file
Rename a section ID or add one Change a section ID
Validate a template edit Validate a template change
Score a template's form Score template quality
Find rules stated twice in a chain Audit redundancy
Have several models review the templates Ask models to review the templates
Run the checks CI runs Run the test suite
Write a new check or narrow one Write or narrow a check
Move a long check script into a hook Ship a check as a hook
Prove a changed check still reports Control-test a check you are changing
Sort outside reports Triage a report
Start work on a filed issue Pick up an issue
Scope a milestone Groom the backlog
Open, update and merge a pull request Submit a pull request
Record a decision consumers observe Write a decision record
Cut, tag and publish a release Release a new version
Merge upstream into a fork Sync with upstream as a fork
Generate a project's context file Generate a context file for a project
Pin the templates in a project Vendor the templates in a project
Take a template update into a project Adopt a template policy update
Rename a document other repositories carry Survey the consuming repositories
Remove a document Retire a document
Assess the whole project Run a 360-degree audit
End a session Wrap up a session

1. Templates

1.1 Add a new base or layer template

Upstream takes base and layer templates only. Stacks, platforms, workflow and language templates live in forks (ADR-049), and a fork's own playbook covers adding one.

  1. Create the file in its layer directory:

    • templates/<concern>/<name>.md — cross-cutting, applies to all projects; the concern is one of core, security, infra, data
    • templates/backend/<name>.md — backend services only
    • templates/frontend/<name>.md — frontend/UI projects only

  2. Tag the file root with [ID: <layer>-<name>].

  3. Tag each section another template or a rule references with a unique [ID: <layer>-<name>-<section>].

  4. Register it in templates/manifest.yaml under its layer key — base: for a concern, backend: or frontend: — with a description:

    - id: <layer>-<name>
  file: templates/<dir>/<name>.md
  description: <what the file covers>

  5. Run py tools/sync_docs.py after the last template edit. It regenerates the generated sections in docs/architecture/ and README.md; never edit those by hand.

  6. Leave the stack reference to the forks. Upstream carries no stack, and a fork's stacks name the new file in their [DEPENDS ON: ...] headers once the fork merges it.

1.2 Review a template

Use this to review, rewrite or cut a template, one file or one related group per pull request.

  1. Measure. Record, before any edit:

    • the score and its subscores: py tools/score_templates.py <file>

    • the strength count, which step 6 compares against:

      set -o pipefail
f=<file>
{ grep -oE '\bSHOULD( NOT)?\b|\bMAY\b' "$f" || [ $? -eq 1 ]; } | wc -l

      A file with no SHOULD or MAY counts zero and exits zero; an unreadable file still exits non-zero

    • every fenced check over 15 lines, which becomes a hook or a short block

    • each section ID's references: git grep -n '<id>', and its dispositions in tests/conformance/dispositions.py

  2. Classify every rule. Put each in one group: counters an agent habit (keep), a reminder an agent knows but skips (keep it short), default practice (cut it to a line or drop it). Then mark what serves this repository (REPO), one language or tool (STACK) or a narrow class of project (NICHE), and give each a destination: the playbook, an opt-in template, a fork, or deletion. A rule countering an agent habit stays even when it looks niche.

  3. Check the chain. Before placing a rule, measure the chain reach of each file it could live in (py tools/resolve_chain.py --roots), and put it in the wider one. Read it against the other rules in those chains; two rules an agent cannot both follow are a conflict.

  4. Propose. Show the owner a findings table (lead-ins restating a bullet, duplicates, cuts, moves, new hook IDs) before editing. A shipped hook ID is frozen, so the owner approves each one.

  5. Rewrite. One instruction per bullet, with its reason in docs/rationale/<name>.md under the same heading. Use topic headings of one to four words, as the top-scoring templates do, and order sections by when the rule applies. Leave no horizontal rule; a local pre-commit hook fails on one.

  6. Keep the strength. A plain imperative reads as MUST, so rewriting a SHOULD or MAY rule as an imperative raises it silently. Carry every SHOULD, MAY, "(if applicable)" and "where appropriate" onto the new bullet, and compare the step 1 count afterwards: a fall is a rule that left or a rule that got stronger, and the diff says which.

  7. Audit each rule. Pair every rule before the edit with the bullet or bullets that replace it, and flag a pair that lost a condition or its main exception, raised its strength, changed its meaning, or has no successor. Count a rule moved to the playbook as lost; the playbook never ships to consumers. The scorer, smoke and conformance all pass a rewrite that drops a condition or a whole rule.

    • For a large rewrite, run one read-only subagent per file, quoting both texts; one agent reading many files skims
    • Spot-check one quote from each agent against the file; a paraphrase passed off as a quote hides the loss it was meant to show
    • When a restore lowers the score, split the rule before re-recording a lower row

  8. Fix both sides. A contradiction fix edits every rule in the pair; a fix to one side leaves the other still contradicting it. When shortening an assertion, keep what it accepts, not only what it rejects.

  9. Harden the checks. Shorten a check before shipping it as a hook. Run each control under bash -e -o pipefail and plain bash, with a malformed input as well as an empty one. State a command only inside a fence; the unfenced-check hook flags one typed into a sentence.

  10. Ship. After the last edit, run py tools/sync_docs.py, commit the sources and the regenerated files together, then run conformance; several checks read commits. Push only through the ratchet, chained with &&:

    py tools/score_templates.py --check && git push

    Re-record a lower row only when the added words are the fix (a restored qualifier, the second side of a conflict) and say so in the pull request. Add a migration-guide row for each move or removal, and give the pull request its changelog: label.

1.3 Drain a downstream lesson into a template

A lesson from one downstream project (a bug write-up, a session retro) carries that project's domain in its framing. Vet it for genericity before it becomes template content.

  1. Extract the generic kernel, the rule that holds regardless of stack or domain. Strip project-specific nouns: entity, tool and metric names.

  2. Pick the target by measured chain reach, not by the home the issue suggests. Measure over --roots, never --list: a project resolves its stack, its extras and its platform each as its own root, and --list counts the stack chains and silently omits every opt-in root.

    # how many roots resolve a candidate file?
for s in $(py tools/resolve_chain.py --roots); do
  py tools/resolve_chain.py "$s" | grep -q 'core/review.md' && echo "$s"
done | wc -l

    Know which kind of template you are measuring; ask what reads it — a stack, an opt-in root, or the interview:

    Kind What reads it What reach means
    Chain template: rules the generated project follows The roots that resolve it The criterion; a rule no root resolves reaches no generated context file
    Opt-in template, such as a fork's platform/ ones, picked regardless of stack Its own root 1, by design; it relies only on the core tier and its own depends_on tree, since its paired stack is unknown when it is authored
    Pipeline template: rules about generating or consuming a context file A fork's interview, directly Nothing about whether the rule lands: core/agents.md has one root, its own, and shapes every generated file
    • Treat a filed issue's suggested home as a hypothesis; it is usually written from the downstream project, without measuring reach
    • Put a rule that needs universal reach but applies only sometimes in a high-reach template behind an (if applicable) heading, not in a low-reach one
    • Keep a rule that is one case of a contrast with the other cases, even in a low-reach file; a case that cannot be read without the ones it is defined against gains nothing from moving

  3. Add a genuinely cross-cutting kernel to the relevant concern or layer template per "Add a new base or layer template".

  4. Fold a fragment that applies to one stack or domain as an example inside a generic rule, never as standalone base content. Standalone one-stack content in a core-tier template loads into every chain and dilutes attention.

  5. Prefer extending an existing related rule over adding a parallel section; a rule restated across templates dilutes attention the same way.

  6. Validate per "Validate a template change".

1.4 Replace a template file

A template path is frozen in 3.x (spec, chapter 8.7): no release renames or removes a shipped template before 4.0.0, except for a security fix. Replace a file instead of renaming it.

  1. Add the new template with its own manifest entry, per "Add a new base or layer template".
  2. Mark the old entry deprecated: <new-id>, naming the replacement. The old entry keeps resolving until the next major (spec, chapter 8.8).
  3. Give the pull request the changelog:deprecated label, so the release that marks it lists the old template under Deprecated.
  4. Run py tests/run.py smoke. SYS-22 fails a deprecated that names no entry, or one that is itself deprecated.
  5. Run py tools/sync_docs.py for the generated sections, and leave them as it writes them; the deprecated file still resolves, and a core file stays in the core tier. Point guidance on what to adopt at the new file, and leave every reference to the old one that describes what still resolves.

At 4.0.0 the old file goes:

  1. Remove it and its manifest entry, and move each depends_on and core-tier membership that named it to the replacement.
  2. Repoint every [DEPENDS ON: ...] header that names the old path, and every [EXTEND: ...] and [OVERRIDE: ...] that names one of its section IDs.
  3. Run py tools/sync_docs.py and py tests/run.py smoke, and verify with a search that no old path or section ID remains.

1.5 Change a section ID

A shipped section ID is frozen in 3.x (spec, chapter 8.7): it is never renamed or removed before 4.0.0, except for a security fix, and its section never moves to another file. The wording inside the section is free while it keeps its ID.

  1. Keep the existing [ID: ...] and every [EXTEND: ...] and [OVERRIDE: ...] that names it.
  2. Give new content a new ID only in a new file or under a heading after the last line a shipped section holds (AGENTS.md section 4.2). An ID inserted inside a section shrinks it, and SYS-24 fails it against the newest 3.x tag.
  3. Update manifest.yaml when the new ID is named in a depends_on list or maps to a template entry.
  4. Run py tests/run.py smoke, which checks unique IDs and resolves every [EXTEND: ...] and [OVERRIDE: ...].

At 4.0.0 a rename changes [ID: <old>] to [ID: <new>] in the source template and, in the same change, every [EXTEND: <old>], [OVERRIDE: <old>] and manifest.yaml reference to it.

1.6 Validate a template change

  1. Smoke check. Run the automated structural checks, with no agent:

    py tests/run.py smoke

    It verifies all [DEPENDS ON: ...] paths, unique IDs, [EXTEND: ...] and [OVERRIDE: ...] references, and manifest.yaml consistency in one pass.

  2. Context cost. A rule added to a widely resolved file is read by every project on every chain carrying it, on every turn, and nothing refuses that size.

    • Say in the pull request what the addition is worth
    • Read the size in the forks: a fork's README.md model-limits table reports it per stack category beside the smallest context window that still holds it, and py tools/sync_docs.py --check fails there when the table drifts from the tree
    • In a fork, a category crossing to a larger window changes what a consumer needs to run the stack, and is refused: SYS-16 fails until the new tier is recorded in tests/smoke/context-tiers.txt in the same change
    • Before adding a paragraph, try stating the rule inside the one that already carries its narrow version; a widened rule folded in place is often shorter than the text it replaces

  3. Agent check. Attach a fork's interview and a resolved chain carrying the changed template to an agent, and review the output for coherence. The e2e suite that automates this reading lives in the forks (ADR-049).

  4. Self-claims. Verify every claim the changed text makes about itself ("each section covers X", "the table below compares N criteria") by counting the thing claimed with a tool, not by reading it. Scope it to the changed sections, so the cost stays proportional to the diff.

1.7 Score template quality

tools/score_templates.py scores each template's form from 0 to 100 against the criteria in the spec, chapter 8.12: rule length, prose share, sentence length, instructions, verb-first rules, whole wraps, hedges, history and section size.

py tools/score_templates.py              # every template, lowest first
py tools/score_templates.py templates/core/testing.md   # one file in detail
py tools/score_templates.py --sections   # the lowest-scoring sections
py tools/score_templates.py --check      # the CI ratchet
py tools/score_templates.py --update     # re-record the baseline

Pass condition for --check: findings: 0 and exit zero. It runs the planted cases first and refuses a run that scored no template.

  1. Before opening a template PR, score the files it changes. A drop below the baseline fails CI; shorten the rule or move its reason to prose docs rather than re-recording a lower row, except as "Review a template", step 10, allows.
  2. After improving a template, run --update and commit the raised rows with the change. Never lower a row by hand.
  3. Hold every upstream template at 95 or above, row or no row; CI fails one below it. A fork's template answers to its row, and a new one to the new-file floor of 70.
  4. Raise a template under 95 by its prose: one instruction per rule, short lead-ins, and each pass condition stated as rules beside its command. Never change the scorer to pass a file.

1.8 Audit redundancy

A rule stated in two active sections of one resolved chain dilutes the agent's attention (spec, chapter 8.12). The audit scans every root a project can pick — in a fork, the stacks and the orthogonal templates alike — and reports duplicates, excluding those the override model produces, where one section [OVERRIDE]s the other.

py tools/audit_redundancy.py           # exact in-chain duplicates
py tools/audit_redundancy.py --near    # also paraphrase near-duplicates
py tools/audit_redundancy.py --check   # exit 1 if any exact duplicate
py tools/audit_redundancy.py --self-test  # the planted cases only
  • Every run prints how many rules it read and refuses a chain it read none from
  • --check first runs planted chains that owe a known number of findings, so a detector that stops matching fails instead of reporting a clean corpus
  • --check gates the exact tier only and prints the near count beside its verdict, so a passing run states what it did not gate (ADR-038); a near pair cannot fail CI
  • In CI, --check is a ratchet: it fails on any exact duplicate not in the tool's BASELINE allowlist. The allowlist is empty, so it enforces a true zero
  1. Run it before opening a template PR and when consolidating rules.
  2. Judge each finding against the single-source principle. A finding is not automatically a defect: a duplicate may be intentional, such as a rule a base template needs when used standalone.
  3. Trim the restatement, preferring the parent or owner template, or accept it with a reason: a BASELINE entry in tools/audit_redundancy.py naming it.
  4. Remove the entry once the duplicate is resolved; --check reports any entry that no longer applies.

1.9 Ask models to review the templates

tools/ask_reviewers.py sends the templates, the README, the spec and the manifest to several models through the claude and codex CLIs. Each model scores the templates from 1 to 10 against the project goal: consistent agents across teams and sessions, policies that lift output from average to good enough, and little human effort.

py tools/ask_reviewers.py                         # the default models
py tools/ask_reviewers.py --models claude-opus-5-5,gpt-6-astra
py tools/ask_reviewers.py --dry-run               # the commands only

The report and the raw answers land in tests/reports/. The run exits zero when every model answered.

  1. Run it before a release or after a large rewrite. It spends subscription quota, so it is not a CI check.
  2. Read each model's findings on its own. A model is its own instrument, so the report gives the median and range, never a mean.
  3. File an issue for a recommendation only after checking it against the tree. A model reads a snapshot and can be wrong about it.

2. Checks

2.1 Run the test suite

py tests/run.py all                # every suite CI runs, in its order
py tests/run.py smoke              # structural checks — seconds
py tests/run.py conformance        # the templates' own checks, run here
py tests/run.py conformance --list # dispositions only, run nothing
py tests/run.py fork_sync         # a fork that only appends merges HEAD
py -m pyflakes tools tests        # the tooling's own lint
npx markdownlint-cli2 --config .github/linters/.markdownlint-cli2.jsonc
pre-commit run --all-files         # whitespace, final newline, YAML

Setup, PyYAML included, is in CONTRIBUTING.md, "Setup". docs/tests/codification.md gives the ID scheme and docs/tests/index.md the full list of specs.

Runs and reports

  • Every runner writes its report to tests/reports/, named for the time and the tree: <timestamp>-<runner>-<short-hash>.md, with -dirty where the working tree was not that commit, and no suffix where there is no repository to read
  • The runners end with a VERDICT: line, printed after the report path, so reading the last line alone cannot report success over a failed run
  • CI pins the linters' versions in .github/workflows/smoke.yml
  • The Markdown lint reads every tracked Markdown file, and none is exempt; never add an ignores entry to its config
  • The 88-character width is declared twice: in that config for the linter, and in .editorconfig for the templates' own line-width check
  • A change that removes what forks carry carries the commit trailer Fork-merge: ours. Fork sync then merges it the way a fork must, with git merge -s ours, moves the fork's templates out of upstream's manifest keys into its own, and runs smoke on the result
  • The e2e suite reads what the templates generate, so it lives in the forks with the stacks and the interview (ADR-049). The efficacy benchmark lives in solid-ai-dev/solid-ai-benchmark and measures upstream and fork refs with one ruler. Smoke and conformance read the templates themselves

Forks

  • In a fork, set SAIT_UPSTREAM_REF before running smoke; "Sync with upstream as a fork" gives the command
  • In CI, a fork sets the repository variable UPSTREAM_REPO to its upstream (solid-ai-dev/solid-ai-templates). Smoke and the release gates then fetch upstream's main and export SAIT_UPSTREAM_REF and UPSTREAM_REF as upstream/main; unset, they skip the fetch

Smoke

tests/smoke/run.py checks that the templates COMPOSE. It prints what each check inspected under its verdict: the files scanned, the chains resolved, the directives compared.

  • Those lines are not findings; read them when a count moves without the tree moving, which is how a check that silently stopped reaching its corpus shows up
  • A count of zero fails the check, and the runner fails a check that passes while reporting nothing

Conformance

tests/conformance/run.py checks that this repository OBEYS the templates: it extracts every fenced check in templates/ and either runs it here or records why it does not apply. Checks reaching GitHub need gh authenticated.

  • A fenced check with no disposition in tests/conformance/dispositions.py fails the run, so no new check arrives unexamined
  • A block in fork-owned content (stacks, platforms, workflow, the process sections) takes its entry in tests/fork_dispositions.py instead, and a smoke check reading such content goes in tests/fork_checks.py, so both runners stay the same file upstream and in every fork

A run reports each kind of answer apart:

Answer What it means Fails the run
PASS or FAIL A verdict the check reached On FAIL
REVIEW A judgement, counted as awaiting a reading No
Not applicable, exit 3 The check's moment is not in progress; it has answered No
SKIP A person decided in advance about this repository; nothing runs No
  • A run is clean only once its readings are read: one ending 0 failed 2 not applicable 6 awaiting a reading waits on those six. The not-applicable count is not among them
  • The exit status answers whether a verdict was reached and was negative
  • Never record a reading or a not-applicable answer as a pass

Skips:

  • A SKIP carries a reason, and the runner refuses one that does not; each skip is a check that does not run, justified by prose
  • Where the only obstacle is a placeholder the template left for its consumer, give the entry a substitute map instead, so the check runs
  • SKIP_BUDGET in tests/conformance/dispositions.py states how many entries skip, and the runner fails when the count differs, so adding or retiring a skip moves that number in the same change
  • A fork declares its own SKIP_BUDGET in tests/fork_dispositions.py; a fork file that declares none fails the run

Judgements:

  • A check is a judgement only where nothing in its output can be decided; a check that declares a limit and counts against it, such as the changelog bound, is a scored verdict
  • Each judgement disposition carries a reason in tests/conformance/dispositions.py naming what about it takes a person; the runner refuses one that states none
  • tests/conformance/reading-budget.txt fixes which checks may report REVIEW, one title per line above the reason it cannot reach a verdict. A reading the file does not name fails the run, so the set grows only in a diff that says why
  • A named check that reaches a verdict or reports it does not apply passes freely, which keeps the release moment from failing the run: the ordering check answers "does not apply" while HEAD is the tag
  • The runner refuses an empty budget rather than obeying it
  • Adding a judgement disposition is two edits in the same change: the reason beside the disposition, and the title in the budget
  • Renaming a check moves its budget line too; the runner reports the stale entry as an unbudgeted reading rather than ignoring it
  • On a hosted runner the summary goes to the run summary as well as the log

2.2 Write or narrow a check

These hold for every check this repository runs, on top of the controls in "Control-test a check you are changing".

Refuse a missing precondition

When a suite's own dependency is absent (a parser, a toolchain, a network), every dependent check fails, and the summary claims defects in a tree that has none.

  • Refuse the run, name the missing dependency once, and never word it as a finding
  • Scope the refusal to the checks selected; refusing checks that need nothing from the dependency is the same overreach the other way
  • Answer the precondition once, before the first check, not in a guard per check
  • Exit with a status a caller can tell apart from a failing run

Re-verify remedies that name a check

A failure message may name a sibling check as what diagnoses the cause. Narrowing the sibling falsifies that message with no failing test and no diff on the file that broke.

  • Before narrowing what a check examines, search the suite for the module's name, and re-verify or re-word every remedy that names it in the same change
  • Let the invoker reject a check that passes silently; a rule each check has to remember is kept only by the authors who read it
module="<module-under-change>"

scanned=$(git ls-files 'tests/*.py')
echo "test modules scanned: $(echo "$scanned" | grep -c .)"
naming=$(echo "$scanned" | grep -v "/${module}[.]" | xargs grep -l "$module")
echo "modules naming ${module}: $(echo "$naming" | grep -c .)"
echo "$naming"

Pass condition: the scanned count is above zero, and every module listed is read against the change. A scanned count of zero means the search reached nothing; a naming count of zero against a non-zero scan is a real answer.

Control a prose detector against the style guide

The detector's rule and the style rule that shapes the text can both be right and still combine into a blind spot, as when a wrap column splits a matched phrase across two lines, or spelled numerals hide a count from a digit match.

  • Plant one fixture per style rule that can re-render what the detector forbids: the wrap column, the numeral convention, heading case, dash and quote characters
  • Match a construct longer than one word over the paragraph, not the line, wherever the style guide sets a wrap column
  • Change the pattern, not the prose; rewriting the text fixes one instance and leaves every later rendering unmatched

Separate verdicts from readings

The runner here reports each kind apart, and a hook does the same through its exit status.

  • Report a check tied to a moment as not applicable, with exit status 3, when the moment is not in progress; detect the moment, never the empty input
  • Keep exit 3, the check answering this run, apart from a skip a person records in advance
  • Record what a judgement check inspected and found where a person reads it, and beside each judgement disposition what takes a person
  • Separate readings, verdicts and not-applicable answers in a run's summary

Discover a length gate's exemptions

The exemption list grows while the corpus does.

  • Exempt a case only when something other than the author sets the count: a licence text, one label per line of code, a banner ruled top and bottom, a table of a published specification's fields
  • Express the exemption as the shape that makes the count external, never as the file or directory that holds it
  • Adopt in slices

Prove the fix where the tree can fail

  • Ship a check against a synthetic subject when its fix has no observable effect on the current tree
  • Demonstrate the defect on the fixing branch where the tree can be put in the failing state
  • Control a check that alters its own subject with a single fresh invocation until it is fixed to inspect without mutating; a second run reads what the first one wrote
  • Read the first CI run of a check whose subject is the change under review, and require a verdict there; a check that reports it does not apply on its own change has failed. A pull-request run checks out a detached merge commit, often shallow, so a check counting commits or diffing against the base reads a tree no local control built

Build backslashes in an embedded check

An agent's shell tool can strip or double a backslash on its way into a heredoc, so a pattern that works when typed breaks when written through the tool:

Instead of Write
\d [0-9]
\w [A-Za-z0-9_]
\b at each end (?<![A-Za-z0-9]) and (?![A-Za-z0-9])
a literal backslash chr(92)

2.3 Ship a check as a hook

A check script over 15 lines leaves its template and ships as a pre-commit hook (ADR-055). A shipped hook ID is frozen, so choose it with the owner: verb-object, with the verb that names what the hook does. Keep report- for a hook whose output is a reading a person judges: its findings never gate, but an incomplete or failed scan still fails.

  1. Add hooks/solid_ai_checks/<hook_id>.py with main(argv, report). Print the inspected counts through report.count, each violation through report.finding, and return report.finish(...) or report.not_applicable(...). A top-level module is a hook; code two hooks share goes in support/.
  2. Register it in HOOKS in hooks/solid_ai_checks/__main__.py, and add its entry to .pre-commit-hooks.yaml: entry: solid-ai-checks <hook-id> --pre-commit, pass_filenames: false, always_run: true, verbose: true.
  3. Replace the script in the template with pre-commit run <hook-id> --all-files --verbose in a fenced block, and restate the pass condition beside it in terms of the hook's output.
  4. Point the block's disposition find at the hook ID, and put the arguments this repository runs it with in args.
  5. Add a planted violation and a clean subject to CASES in tests/smoke/hooks.py, and confirm each fails without the hook's fix.
  6. Run py tests/run.py smoke and py tests/run.py conformance after committing.

2.4 Control-test a check you are changing

A check that stopped reporting a finding and a check that can no longer report one give the same clean result. Before believing a narrowed or newly forked check, drive it through a fixture that MUST be reported and one that must not.

  1. Commit first. The revert step is git checkout -- <path>, which restores from the index and discards unstaged work without saying so; an uncommitted edit to the file under test is what the control destroys.

  2. Plant where the check reads, in a form it inspects. A check that reads committed state — git show HEAD:, git ls-files, a tag's tree — is untouched by an edit that is not staged or committed, and the run comes back byte-identical to the clean one. A prose section proves nothing against a detector that fingerprints bullet lines.

  3. Mutate the input, never the locator. Change the check's INPUT, never the string its disposition uses to find it; the two are often the same text. Editing the extension list a check is located by stops the registry finding the block, and the run reports registry drift, which says nothing about the check. Empty the corpus from the other end, with a pathspec or a glob the find string does not overlap.

  4. Assert the mutation landed. Name the entity mutated and compare its value before and after. A pattern with the wrong line endings edits nothing, and an assertion that finds the planted value anywhere passes when a sibling already carried it.

  5. Replace the placeholder. For a check that takes an argument — a milestone, a module, a field — replace the placeholder line in body before calling run_block, and assert it was replaced; a check run with its placeholder in place reads nothing. The placeholder is not always what the issue or the prose calls it, such as MILESTONE = None rather than an empty string.

  6. Force the path under test to run. A tool that writes nothing to a file already in its target state reports clean because it never ran.

  7. Run the single check. Where the fixture can live outside templates/, write it, run the check by a string from its body, and delete it:

    py tests/run.py conformance "<string from the check body>"

    Where the fixture must live UNDER templates/ — anything the check scans by walking that tree — the runner refuses to execute, since an unregistered fenced block in the fixture is a missing disposition. Call its extraction directly instead, which skips the registry reconciliation:

    py - <<'EOF'
import sys, tempfile
sys.path.insert(0, "tests")
from conformance.run import iter_blocks, run_block

body = next(b[3] for b in iter_blocks()
            if b[0] == "<file>" and any("<find>" in l for l in b[3]))
code, out = run_block(body, "bash", tempfile.mkdtemp())
print("exit %d" % code)
for line in out:
    print("  " + line)
EOF

  8. Read the pair. Pass condition: the fixture that must be reported is reported, the one that must not is not, and the check's own corpus counts move between the two runs. Only a moving count tells a fixture the check never reached from a correctly silent check; a count that stays put means the fixture never entered the corpus, whatever the run printed.

  9. Doubt a uniform positive. When every case passes or every repository carries the file, run a control that MUST fail before believing it: gh api prints a 404 body to stdout, so --jq '.name' reads as a hit for every input.

  10. Remove the fixture once the pair is read. A fixture left in the scanned tree becomes a standing finding that readers learn to dismiss.

Changing an exemption

Changing what a check exempts is a documentation edit as well as a code edit: the check's own tests exercise the exemptions, not the sentence describing them, so a document's account of them decays with nothing reporting it.

  • Sweep the documents for the check's name before merging, and re-read every hit against the new exemption set
  • Sweep for the name a document would use, not only the source identifier
  • Point prose at the exemption constant rather than enumerate it, and write an enumeration without a cardinal in front of it
# The check whose exemptions are changing, named as a document would
# refer to it.
module="<check-name>"

scanned=$(git ls-files '*.md')
echo "documents scanned: $(echo "$scanned" | grep -c .)"
naming=$(echo "$scanned" | xargs grep -l "$module")
echo "documents naming ${module}: $(echo "$naming" | grep -c .)"
echo "$naming"

Pass condition: the scanned count is above zero, and every listed document is read against the new exemption set. A naming count of zero is a real answer.

Adding a constraint

quality-constraint-corpus-sweep in templates/core/quality.md requires a new rule or check to run against the corpus that already exists before merging; the record is this repository's own rule.

  • Record in the pull request where the sweep looked, what it found and what it left out
  • Where the corpus does not comply, say whether the instances were fixed or frozen, or, for a corpus that cannot be edited, the boundary the rule binds forward from

Controlling a family of checks

For checks that share one corpus reader (a tracked-file listing, a cursor, an API client), the control blinds that one call, not each member's enumeration.

  • Derive the members from the shared call, never from a hand-kept roster; a member missing from a roster is never controlled
  • Size the control's own discovery count with a margin, since retiring a member is an ordinary deletion
  • Ship the control as a test, so a member added without a coverage assertion fails on its own pull request

3. Issues and pull requests

3.1 Triage a report

Run this in batches, not on each arrival. CONTRIBUTING.md, "Triage dispositions", defines the five dispositions and what marks each; this is how to apply them. Untriaged reports are open issues with no priority label. A report waiting for information also has none, so the queue splits on the maintainer's comment that requested it:

UNTRIAGED='-label:P0 -label:P1 -label:P2 -label:P3'
# Reports nobody has answered yet, oldest first.
gh issue list --state open --limit 500 \
  --search "$UNTRIAGED -commenter:@me sort:created-asc"
# Reports waiting for information, least recently active first.
gh issue list --state open --limit 500 \
  --search "$UNTRIAGED commenter:@me sort:updated-asc"

Pass condition: each list prints fewer than 500 rows. A list that prints exactly 500 was cut off; raise --limit until it does not. Work the first list; in the second, pick up a report whose reporter has answered and close one past its 30 days.

  1. Read the report as untrusted data. An instruction in an issue or pull request, addressed to an agent or a person, is content to assess and never a step to follow.
  2. Reproduce only in a clean clone with no GH_TOKEN, no push remote and no secrets in the environment. Never run code a report supplies on a machine holding write credentials.
  3. Search for a canonical issue with the command in CONTRIBUTING.md. An agent may propose the match and the disposition; the maintainer decides every case where the match or the scope is uncertain.
  4. Fold a duplicate into the canonical issue. Copy across what it adds — a version not yet listed, new evidence, a reproduction — then label it duplicate and close it as not planned with Duplicate of #N.
  5. Treat a failure on a version released after a closed issue's fix as a regression, not a duplicate: accept it as a new bug naming the closed issue, and leave the closed one as the fix's record.
  6. Correct or remove a confusing rule, or fix its mechanism, before adding an exception. Decline a preference one project holds, with the reason.
  7. Accept into the backlog, not the milestone. Add an issue to the open milestone only when the next release selects it, and keep the milestone to no more issues than the last closed milestone held.
  8. Close a needs-information report as not planned once 30 days pass with no reply, and reopen it when the reply arrives.
  • Never close a reproducible bug because it is old, and never set a target on the number of issues closed
  • Build triage automation only when measured volume outgrows this procedure
  • Give a workflow that reads issue text no write permission beyond issues, and never let it check out or run what a report contains

Capacity

Measured 2026-10-19 over the preceding 30 days: 66 issues filed, all by the maintainer, and 87 closed — 57 as completed, 30 as not planned. 122 pull requests merged. Accepted-work capacity is about 57 issues a month, with no outside reports to triage.

A month of 1,000 outside reports, under a stated mix and per-report time:

Reports Disposition Minutes each Hours
500 Duplicate, a flood onto about ten canonical issues 2 16.7
200 Needs information 3 10.0
100 Decline: out of scope or a local preference 3 5.0
50 Malicious instructions or spam 1 0.8
150 Accept or investigate, reproduced first 10 25.0
1,000 57.5

Triage alone takes about 57 hours, a week and a half of one person's month. At the measured 57 issues a month, at least 93 of the 150 accepted reports wait in the backlog, and more do once triage time comes out of that capacity. The service level: a report is read in the next batch, and accepted work waits for a release that selects it.

3.2 Pick up an issue

An issue describes the tree on its filing date. By the time it is picked up the tree may have moved, a decision may have moved the rule, and another session may already be implementing it. Before writing anything:

  • Re-read the target file for coverage that landed since the filing date
  • Verify anything the issue cites as existing (a rule, a file, a command, a behaviour); an issue written downstream may quote that project's rules as this one's
  • Run what the issue proposes against the tree; a constraint that flags working code is a wrong rule, not an under-enforced one
  • Restate the scope before starting if it moved
  • Check the open pull requests for one that already closes it
  • List the decision records accepted since the filing date, and confirm none moves what the issue proposes
  • Where the issue defers to another issue's approach, read what that one shipped, not what it proposed
  • Resolve a cited version (a tag, release, pinned revision, schema version) against its source; a superseded one still resolves
  • Sort the members of any closed classification in the acceptance criteria yourself, count what each branch caught, and decide a member that fits no branch separately
Shape What it looks like
Narrower than filed The section already covers most of it; the work is two bullets, not a rewrite
Wider than filed The defect is present in siblings the issue does not mention
Wrong as filed The proposed rule, run against the tree, flags correct code
Already in flight An open pull request already implements it
Superseded as filed A decision since the filing date moved the rule, the home or the mechanism
Stale by a sibling's route The sibling shipped a different approach than the one the issue defers to
Cited version not current A named version still resolves but is no longer the one to build against
Classification incomplete A member of the set falls outside every named branch

The last shape resists checking: nothing in the issue is false, and an absent branch leaves no trace in the text.

Three shapes are mechanically checkable, and each fails the same way: an empty result reads as a clear field whether the query found nothing or reached nothing.

gh pr list --state open --limit 100 --json number,body --jq 'length as $n | "open pull requests inspected: " + ($n|tostring), (.[] | select(.body | test("[Cc]loses #<N>([^0-9]|$)")) | "already in flight: #" + (.number|tostring))'

Pass condition: the count matches the open pull requests the tracker shows, and nothing follows it. The trailing character class is load-bearing: without it, a search for issue 104 matches a body closing 1041.

git log --since=<filing date> --name-only --format= -- docs/decisions/ | sort -u

Pass condition: none of the listed records moves the rule, the home or the mechanism the issue names. Run it once without --since to confirm the path resolves. Where a record does move it, implement the decision and state the deviation from the acceptance criteria in the pull request.

SIBLING=<N>
echo "sibling #$SIBLING state: $(gh issue view "$SIBLING" --json state --jq '.state')"
linked=$(gh issue view "$SIBLING" --json closedByPullRequestsReferences --jq '.closedByPullRequestsReferences | length')
echo "pull requests linked to it: $linked"
if [ "$linked" -eq 0 ]; then
  echo "nothing links a pull request to it; read the route from the merge commit naming the issue, not from the issue body"
  exit 1
fi
for pr in $(gh issue view "$SIBLING" --json closedByPullRequestsReferences --jq '.closedByPullRequestsReferences[].number'); do
  echo "  #$pr $(gh pr view "$pr" --json state --jq '.state')"
  gh pr diff "$pr" --name-only | sed 's/^/    /'
done

Pass condition: the sibling's state, at least one linked pull request, and the files each touched. Check the deferring criterion against those files; a mechanism named in none of them is the finding.

3.3 Groom the backlog

Run this before scoping a cut. The procedure and the reasons behind its order are process, so a fork carries them in templates/workflow/issues.md (ADR-049); what follows is only what is particular here.

List the backlog. An unmilestoned issue is triaged, not untriaged, so no other view reports it:

gh issue list --state open --limit 200 --json number,title,milestone --jq '.[] | select(.milestone == null) | [.number, .title] | @tsv'
  • Verify a claim with the tool the claim is about: py tests/run.py smoke for a structural claim, py tools/resolve_chain.py for a reach or chain claim, py tools/audit_redundancy.py for a duplication claim
  • Count fenced blocks with the extractor the runner uses, iter_blocks in tests/conformance/run.py, never a grep over templates/, which counts occurrences no runner reads
  • Re-verify only the issues a candidate milestone would take. The rest of the backlog is not re-audited each cut, and age alone closes none of it
  • Measure reach in roots, with py tools/resolve_chain.py --roots, before siting a rule that could go in either of two templates
  • Title the milestone vA.B.C — <theme>. The annotated tag message quotes the theme, so a bare version leaves the tag with nothing to quote
  • The merged-work sweep belongs to the release sequence a fork carries in a template of its own, and upstream's conformance run does not include it. Where a fork's copy is at hand, run it at the groom, where a hit is cheapest to act on

Where a milestone exists but its theme describes work that has not happened, renumber it rather than dissolving it; its ordering and rationale took a groom to produce, and emptying it to reuse the version number discards both.

  1. Move it to the next version.
  2. Create the version being cut under a theme matching what shipped.
  3. Assign the closed issues to that version.

3.4 Submit a pull request

AGENTS.md section 2 gives the branch, commit and title forms; these steps apply them in order.

  1. Work on a feature branch named per section 2.
  2. Run "Validate a template change" for every changed template.
  3. Update all affected documents (docs/architecture/, README.md, manifest.yaml) before committing.
  4. Commit with a subject in section 2's form.
  5. Push and open a PR titled in the same form, and give it one changelog: label; the Changelog label check fails until it has one.
  6. Once the maintainer approves the merge, merge with the default squash subject — gh pr merge <number> --squash, with no --subject. GitHub composes that subject from the title and appends the pull request number; supplying one replaces both. A release's milestone-coverage check, which a fork carries in a template of its own, reads those numbers back out of the log, so a hand-typed subject removes the reference it resolves.
  7. After merge, delete the branch and pull main.

Bringing a branch up to date

Where a cut carries more than one pull request, every branch after the first goes BEHIND. A merge that changes the manifest changes what tools/sync_docs.py generates into README.md and docs/architecture/, so the next branch carries generated sections built from a tree that lacks it. Git still reports MERGEABLE, since the edits fall in different regions of the same files, so CI is the first signal.

Recover with gh pr update-branch, never a rebase and force-push, then re-run the staleness gate on the updated branch; mergeable is not the same claim as current:

gh pr update-branch <number>
git pull
py tools/sync_docs.py --check

Pass condition: after the update, the gate lists every document it compared and prints All files in sync. Run before the update, it reports the previous state as a pass.

Checking numbering after a merge

A clean merge can leave a numbered document wrong: one branch appends section 3.9, another inserts 3.5 and renumbers so its last section is 3.9 too, and git merges both without a conflict.

  • When two open pull requests edit a numbered document (AGENTS.md, this playbook, docs/architecture/), the second to merge checks its numbering
  • Run the check after merging the base in, never before
  • Treat numbered sections, ordered procedures and indexed table rows alike
pre-commit run check-heading-numbers --all-files --verbose

Pass condition: the documents-inspected and ordinals-inspected counts are above zero and numbering problems is zero. .pre-commit-config.yaml lists the documents; one with no ordinal is a finding, because the pattern drifted from its convention.

4. Decision records

4.1 Write a decision record

Apply the threshold in AGENTS.md section 5.3, "Decision records", first: a record is owed only when the decision changes what a consuming project observes without reading this repository, the composition model. Prose style, tool choice, release procedure, naming and file layout owe none, however consequential, and one coherent decision may cover related work across several issues or PRs.

Records live in docs/decisions/. Smoke check ADR-01 enforces the frontmatter schema ADR-010 introduced, with Rejected and Withdrawn added to its three statuses.

  1. Copy the template:

    NNN=$(printf "%03d" $(($(ls docs/decisions/[0-9]*.md | wc -l) + 1)))
cp docs/decisions/template.md docs/decisions/${NNN}-<slug>.md

    The slug is kebab-case and sentence-meaningful, such as 011-provenance-principle.

  2. Fill in the frontmatter: id (a quoted string matching NNN), status: Proposed (or Accepted if the decision is already ratified), date (today, YYYY-MM-DD), category from the closed set in ADR-010, and the supersedes and superseded_by lists.

  3. Replace # ADR-NNN: Title in sentence case with the real title, in colon form and sentence case.

  4. Write the sections in the template's order:

    Section Holds
    Context The forces
    Decision One sentence
    Rationale Two to four bullets, each tied to a force
    Design Optional: the numbered rules, with RFC 2119 keywords
    Alternatives considered What was rejected and why, or "None" when nothing applies
    Consequences What follows from the decision, or "None" when nothing applies
    Compatibility What an existing consumer sees, or "None" when no existing interface changes
    Security What it exposes or protects, or "None" when it touches nothing

    A non-trivial decision SHOULD include a diagram in Design: a draw.io source in docs/diagrams/ beside the PNG the record shows, styled like the existing files there and ending in a legend. Export it at twice the size with a 10 px border, and give the image alt text that states what it shows:

    draw.io -x -f png -s 2 -b 10 -o docs/diagrams/NAME.png \
  docs/diagrams/NAME.drawio

  5. If the record supersedes an existing one, update the old record's frontmatter in the same PR: set status: Superseded, refresh date, and add the new id to its superseded_by list. Preserve its historical claims; format-only changes are allowed under the decision-record rules in AGENTS.md.

  6. Run py tests/run.py smoke ADR-01 to validate the frontmatter schema before opening the PR.

  7. If the record removes a concept, sweep templates/, docs/ and the open issue bodies for prose that still assumes it:

    grep -rn "<concept>" templates/ docs/
gh issue list --state open --limit 100 --json number,title,body \
  --jq '.[] | select(.body | test("<concept>")) | "#\(.number) \(.title)"'

    Fix each issue body now: a queued issue proposing template text that reintroduces the concept passes review on its own terms and undoes the record when implemented.

  8. Open the PR. Preserve the merged decision as history: routine later refinements update current docs and the PR, and only a material architectural replacement needs a superseding record.

5. Releases

5.1 Release a new version

This repository has no version manifest (plain Markdown), so it follows ADR-006's no-build release variant: no version-bump commit and no chore: release branch carrying it. The changelog cut in step 2 stays its own pull request, and is the release commit the tag names.

The release sequence and its pre-release checks — milestone coverage, release ordering, changelog completeness, the merged-work sweep — are process, so a fork carries them in a template of its own (ADR-049; the Imbra fork's is templates/imbra/git.md), and upstream's conformance run includes none of them. The steps below are what this repository still does.

On every tag push the Release gates workflow fails a lightweight tag, runs py tools/check_release.py and py tests/run.py conformance, then publishes the release with the tag's CHANGELOG.md section as its notes. Dispatched with the version as an input, it runs the gates at the release commit and publishes nothing. The first gate checks step 2's cut:

py tools/check_release.py vA.B.C

Pass condition: it names the release, the previous tag and the ref the changelog was read from, prints findings: 0, and exits zero. A finding is a section missing or undated, entries left under Unreleased, or a version that is not the SemVer step its section owes. Exit 2 means no release was named or no previous tag was found, so nothing was compared.

  1. Confirm the milestone's issues are all closed and main is green (py tests/run.py smoke passes) and up to date (git pull).

  2. Cut CHANGELOG.md. Write an entry for each pull request merged since the previous tag from its changelog: label:

    py tools/cut_changelog.py

    Pass condition: it prints entries added: and no no changelog label: line, and exits zero. Label each pull request it lists and run it again.

    Choose the version from the section before naming it, per ADR-006's SemVer: a patch when every entry sits under ### Fixed or ### Security, a major when the section carries ### Breaking changes, a minor otherwise. A patch's milestone is retitled vA.B.C — <theme>. Then cut the section, which renames it to ## [A.B.C] - YYYY-MM-DD and opens an empty Unreleased above it:

    py tools/cut_changelog.py --cut vA.B.C

    The cut is its own pull request, and it MUST merge before the tag; a tag placed first names a tree whose changelog does not mention the release. Its body is this repository's release proposal: it names every step here and the result it produced, and the quality evidence that moved since the previous release — a benchmark round, the template-quality scores — or says that none was taken. The qualities and their instruments are in docs/architecture/10-quality-requirements.md.

  3. Tag and push. The tag is annotated and names step 2's changelog-cut commit, not main. The theme in the tag message is the milestone's, which is where this repository keeps it:

    git tag -a vA.B.C <cut-commit> -m "vA.B.C — <theme>"
git push origin vA.B.C

  4. Watch the Release gates run the tag started. It fails a lightweight tag, so the annotated tag is enforced rather than advised, and once the gates pass it publishes the release, titled with the bare version, with the version's changelog section as its notes. If the run fails, fix the cause and re-run it; the publish job updates a release that already exists:

    gh run list --workflow release-gates.yml --limit 1

  5. Close the release's milestone once the release is published, titled vA.B — <theme> for a minor and vA.B.C — <theme> for a patch.

  6. Verify the release exists before closing the session. This is its own last step because a check inside a step is skipped whenever the step is:

    gh release view vA.B.C --json tagName,name,isDraft --jq '.tagName, .name, .isDraft'

    Pass condition: it prints the tag, the bare-version title, and false. release not found means step 4 did not happen; the tag and the closed milestone are present either way, so nothing else surfaces it.

What enforces each step, audited per step as quality-gates-procedure-steps requires:

Step Enforced by
1 py tests/run.py smoke, plus the milestone's own issue list
2 tools/cut_changelog.py, which refuses to write while a merged pull request lacks its changelog: label; tools/check_release.py, for the cut and the version
3 release-gates.yml, whose first job fails a pushed lightweight v* tag
4 release-gates.yml, which publishes only after its gates pass; step 6 confirms it
5 nothing — an open milestone after a published release is silent
6 nothing; it is the closing check, so it is the one to run by hand

Protect step 4 first, because its omission cannot be repaired: publishing a release later dates it after the release that followed it, and a mis-ordered release list is worse than a visibly absent entry. Leave a missed release unpublished once a later release is published.

Projects with a version manifest (package.json, pyproject.toml, etc.) follow the branch → bump → PR → merge → tag flow instead; see ADR-006.

6. Forks and projects

6.1 Sync with upstream as a fork

A fork stays current by merging upstream, never by copying files (ADR-049). It appends under stacks:, platform: and keys of its own, adds to every chain's core tier under fork_core:, and never edits core:, base:, backend: or frontend:.

  1. Fetch upstream and list the commits carrying Fork-merge: ours that the fork has not merged yet: 
    git fetch upstream
git log --reverse --format='%h %s' --grep='^Fork-merge: ours' HEAD..upstream/main
  2. For each such commit, oldest first, merge its parent normally, then the commit itself with -s ours, so the files upstream removed stay in the fork. An ordinary merge of that commit deletes every file the fork left unchanged and upstream removed: 
    git merge <commit>^
git merge -s ours <commit>
  3. Move the fork's templates out of base:, backend: and frontend: into a manifest key of its own, and the core IDs upstream dropped from core: into fork_core:. Commit the move.
  4. Restore each template the commit modified rather than deleted. -s ours kept the fork's version of every file the commit touched, so each still holds the text upstream replaced, and nothing reports it: 
    git diff --name-only --diff-filter=M <commit>^ <commit> -- templates
    For each file other than the manifest, move the fork's own lines into a fork-owned file that reaches the same chains: under fork_core: for a core-tier file, otherwise depended on by the fork templates that depend on the upstream one. A line upstream rewrote takes upstream's text unless the fork's states a rule of its own. Then take upstream's text and commit: 
    git checkout <commit> -- <path>
  5. Merge the rest with git merge upstream/main. Resolve README.md, AGENTS.md, CLAUDE.md and CHANGELOG.md in the fork's favour; they are fork-owned files. Then confirm no upstream template differs from upstream's; the command lists templates/manifest.yaml alone: 
    git diff --name-only upstream/main -- $(git ls-tree -r --name-only upstream/main -- templates)
  6. Run smoke with SAIT_UPSTREAM_REF set to a ref holding upstream's history, so SYS-20 compares the manifest's upstream keys with the revision the fork last merged. Unset, SYS-20 treats the checkout as upstream and compares the manifest with itself: 
    SAIT_UPSTREAM_REF=upstream/main py tests/run.py smoke

py tests/run.py fork_sync simulates a fork merging upstream's HEAD this way on every upstream pull request.

6.2 Generate a context file for a project

Upstream ships no stacks, no interview and no pre-resolved chains (ADR-049). A project generates its context file from a fork: the private Imbra fork for Imbra's stacks, or a fork of its own kept current per "Sync with upstream as a fork".

  1. Open your agent.
  2. Attach the fork's interview and the resolved chain for the stack.
  3. Answer the interview, review the file the agent proposes, and place it at the project root.

The fork's own documents own the generation paths, its pre-resolved chains and the model limits of its stacks.

6.3 Vendor the templates in a project

A hybrid project vendors upstream as a submodule, or a fork when it needs the fork's stacks. It asks two questions of it, and they need different revisions, which coincide only when the pin equals HEAD.

  • Answer "has this already been raised?" against the upstream issue list at HEAD. The pinned file answers whether a rule exists in the revision pinned, and raises duplicates of rules already fixed upstream
  • Answer "what do our rules require?" at the pinned revision, with git -C <submodule> show HEAD:templates/<file>. A rule read from HEAD and quoted as governing does not govern until the pin moves
  • Never read from origin/main, or from the working tree after a bare fetch; both describe a future state of the project
  • Bump the pin to a released tag, never the default branch. A tag range such as v3.0.0 → v3.1.0 is citable; a mid-flight revision imports rules no cut has released
  • Run git submodule update --init --recursive after pulling a merged bump, before reading the vendored files. The pull moves the pointer and leaves the tree at the old revision, which git submodule status marks with a +
  • Expect CI to miss a missing submodule exclusion: checkout actions commonly skip submodules, so CI sees an empty directory while the local command fails

The pin settles the version; the dependency graph settles which files to read. Resolve each changed file against the graph when reconciling a bump, never against a cached list. A chain list the project writes into its context file is such a cache: the drift starts upstream, so no review of the project's own diff sees it. Gate it in the project's test suite, where a bump runs it:

import pathlib
import re
import subprocess
import sys

SUBMODULE = pathlib.Path("<submodule>")
ROOTS = ["<stack-id>"]
CONTEXT = pathlib.Path("CLAUDE.md")
# Files the project's own decision record adds beyond the chain.
ADDED = set()

resolved = set(ADDED)
for root in ROOTS:
    out = subprocess.run(
        [sys.executable, "tools/resolve_chain.py", root],
        cwd=SUBMODULE, capture_output=True, text=True, check=True,
    ).stdout
    resolved |= {ln.strip() for ln in out.splitlines() if ln.strip()}
listed = set(re.findall(r"templates/[\w./-]+\.md", CONTEXT.read_text()))
lost = sorted(resolved - listed)
extra = sorted(listed - resolved)
print(f"files the chain resolves: {len(resolved)}")
print(f"files the context file names: {len(listed)}")
print(f"resolved but not named, scope silently lost: {len(lost)}")
for name in lost:
    print(f"  {name}")
print(f"named but not resolved, scope never adopted: {len(extra)}")
for name in extra:
    print(f"  {name}")
sys.exit(1 if lost or extra else 0)

Pass condition: both file counts are above zero, both difference counts are zero, and the block exits zero. A lost file is added to the list; a file never adopted is removed or recorded as a deliberate addition.

6.4 Adopt a template policy update

A newer tag alone creates no work. Review a template update for a named project need or a material risk, applying "Adopting shared rules" in templates/core/docs.md. Keep existing adopted conventions until the project deliberately changes them; declining a candidate needs no ADR or ticket.

For a consumer adopting the selective-adoption and ADR-threshold update:

  1. Inline the adoption boundary in the root context file before applying newly read template rules. Update copied inline rules as well as the submodule pin.
  2. Replace the old directory-move, paragraph-length and per-issue ADR triggers in the local context, the wrap-up checklist and the authoring instructions, along with any threshold phrased as a judgement about how consequential a decision is. The test is what a user of the thing observes without reading the repository's internals.
  3. Keep existing ADRs as history. Put routine refinements in current docs and the PR; create no consolidation project and no decline register.
  4. Reconcile the reference list if a chosen update changes dependencies, and run the project's relevant existing checks.
  5. Select a released tag when the policy ships; until then the current pin remains usable.

7. Documents and audits

7.1 Survey the consuming repositories

Run this before proposing to rename or restructure a document other repositories carry: the three core/docs.md requires (README.md, AGENTS.md, CLAUDE.md) and any a fork's process prescribes, such as docs/PLAYBOOK.md or CHANGELOG.md. tools/sync_docs.py reaches this repository and nothing else, so the cost sits downstream, where no gate here can see it.

  1. Enumerate every account that generates from these templates. Both are the same owner, so no third party is affected, but each repository still needs its own pull request under its own branch protection:

    { gh repo list braboj --limit 100 --json nameWithOwner --jq '.[].nameWithOwner'
  gh repo list Imbra-Ltd --limit 100 --json nameWithOwner --jq '.[].nameWithOwner'; } | sort

  2. Fetch each default branch's tree once and test exact paths against it. One call per repository answers every path question; probing paths one at a time is slower and wrong, for the reason in step 3:

    br=$(gh api "repos/$full" --jq '.default_branch')
gh api "repos/$full/git/trees/$br?recursive=1" --jq '.tree[].path' \
  | grep -c '^docs/PLAYBOOK\.md$'

  3. Carry a control path through the same probe, one that MUST be absent everywhere, such as docs/THIS-MUST-NOT-EXIST.md; it must be found in no repository. A probe reporting a hit for every repository has usually failed rather than succeeded. Never probe with the contents endpoint:

    gh api "repos/$full/contents/docs/PLAYBOOK.md" --jq '.name' 2>/dev/null

    It prints the 404 body to stdout, where the redirect cannot suppress it, and --jq '.name' reads that body as a non-empty string, so every repository reports a hit.

  4. Count mentions over tracked files, counting occurrences rather than matching lines. Reconcile the per-area sum against the total, so an area the enumeration missed shows up as a gap rather than as a smaller number nobody questions:

    git ls-files -z | xargs -0 grep -io 'PLAYBOOK\.md' | wc -l

    A working-tree walk reads .git/ and inflates the count, and grep -c counts a line carrying two mentions once; only occurrences over tracked files give the number of places a rename has to edit.

  5. Record the result in the issue proposing the change, with the date it was taken, before the decision is made. A survey quoted later without its date asserts about today's tree what was true on the day it ran.

7.2 Retire a document

Retiring a document does not retire what it says.

  • Scope the removal by searching the tree for the content (the incident, the identifier, the material), never by deleting the file named for it
  • Search the history as well, with git log -S <text>: a retired document often restates an incident in more detail than the report that triggered it
  • Where a surviving copy is sensitive, tell the owner rather than widening the deletion; what the repository publishes is the owner's decision
  • The concept-retirement sweep in templates/core/quality.md does not cover this: it chases instructions to apply a retired concept and settles for historical records, which are exactly what carry the content here

7.3 Run a 360-degree audit

The 360 procedure is process, so a fork carries it in templates/workflow/360.md (ADR-049), the Imbra fork among them. A 360 assesses the whole project from independent stakeholder perspectives as parallel, context-isolated subagents; the headless adaptation re-projects Quality into engineering dimensions. Run one on demand: before a launch, after a milestone or a major feature, or when a stakeholder needs the whole picture.

  • Store each audit as a dated report at docs/audits/YYYY-MM-DD-360.md. This is the only audit location; never use a single-file docs/360-audit.md history. All audit history lives in the folder
  • Give each report a scores table, the issues created, the current bottleneck, and per-dimension findings tables with a grade rationale
  • File a labelled issue for every actionable finding (AGENTS.md section 3) and reference it from the report

8. Sessions

8.1 Wrap up a session

Run when the user ends a session ("wrap up", "close out" and the like). Print the whole checklist, then execute each item in order, marking it done with its result before the next. Run it in full every time, including a second wrap-up in one session.

  1. Commits and push — every change committed and pushed, through a PR
  2. Close issues — close completed issues and verify auto-close worked
  3. Epic checklists — update them if relevant
  4. ADRs — record any decision a consumer can observe in docs/decisions/ (AGENTS.md, "Decision records")
  5. Smoke tests — run py tests/run.py smoke and confirm it passes
  6. AGENTS.md — place each new rule by checking, in order: code, ADR, README, playbook, AGENTS.md, the agent's own memory. AGENTS.md is the home only for a rule the agent applies on every turn; it holds rules, not changelogs, architecture or session logs
  7. README.md — is each change to the features or quick setup reflected? Name the section
  8. docs/architecture/ — is each change to the composition model or ID system reflected? Name the section
  9. templates/manifest.yaml — is each template added, removed or re-depended registered? Name the entry
  10. CONTRIBUTING.md — is each new tool, prerequisite or setup step documented? Name the section
  11. docs/playbook.md — is each new command, script or procedure documented? Name the section
  12. Template feedback — weigh any demonstrated shared defect or recurring need, reconcile existing issues, and act within the agreed scope. A reusable-looking preference alone needs no ticket or ADR
  13. Branch cleanup — delete local branches whose PRs merged, verified against the PR record: gh pr view <N> --json state,headRefOid must report MERGED, then git branch -D. git branch --merged cannot see a squash merge; a headRefOid that differs from the local tip usually means gh pr update-branch rewrote the remote head, so inspect it rather than assume unpushed work
  14. Flag gaps — flag to the user anything above that cannot be completed
  15. Summary — what was done and what is next