Skip to content

Add automated CI check enforcing docs/stability-tiers.md compatibility guarantees #2109

Description

@codeforester

Goal

Make the compatibility promises in docs/stability-tiers.md (stable command flags, versioned JSON schemas, stable finding IDs) mechanically enforced in CI, not just documented as policy.

Background

Base currently merges to main at high velocity (observed ~8 merges within a 90-second window during a single review session), which is healthy for a fast-moving solo project but makes it impossible for an external adopter to tell, from the outside, whether the "stable" label on a command or JSON contract is actually enforced or just aspirational. Nothing in .github/workflows/ currently diffs stable surfaces between a PR and the prior release. This is the automated counterpart to the policy work in #1620 (release stabilization, candidate, and independent-review policy) — that issue defines the policy; this issue is the enforcement mechanism it needs.

Scope

  • Snapshot the stable command surface (documented flags, --format json payload shapes for the schemas listed in docs/stability-tiers.md, and doctor/check finding IDs) as a versioned fixture.
  • Add a CI job that diffs a PR's actual output against that fixture and fails on: removed/renamed keys, changed value types, changed enum meanings, or a reused finding ID — unless the PR also updates the fixture and includes a stability-tiers.md-referenced changelog entry.
  • Document the check and its override path (deliberate breaking change with migration notes) in docs/stability-tiers.md.

Acceptance Criteria

  • A new CI job fails on an intentionally introduced breaking change to a stable command/JSON surface in a test PR.
  • The job passes when the same change is accompanied by an updated fixture and changelog entry.
  • docs/stability-tiers.md documents the enforcement mechanism, not just the promise.

Related: #1620

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ciContinuous integration, tests, automation, or release workflowsenhancementNew feature or product improvementtype:feat

Type

No type

Projects

  • Status
    Triage

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions