Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .claude/skills/checklist-author/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,15 @@ metadata:
3. **本技能的简报是假设,源码才是真相** —— 每个测试项按 README.md「Item anatomy」
的契约写;缺 fixture 记 `blocked`/`knownGaps`,永不伪造覆盖。

## 缺项写成 `planned` 项,不写成注释

sweep 发现「定义要求、平台还没有」的能力,写成该区的 `status: "planned"` 项(title ·
priority · personas,`since: null` 或目标 release,⛔ 无 steps);注释没有 id,派发时指
不到。它是 `coverage.json` 的合法映射目标但⛔ 不算覆盖:只映射到它的种类仍报 UNMAPPED。

**`planned → active` 只由一次判 pass 的运行记录兑现** —— steps/acceptance 随实现 PR 落
地,跑过才翻状态(bump `revision`、追 `history`);⛔ 代码落地就翻 = 账本开始虚报覆盖。

## 规模指引

一轮全量 sweep ≈ 5 个 hunter + 8 个 writer agent。范围化的问题(「X 有测试吗?」)
Expand Down
4 changes: 4 additions & 0 deletions .claude/skills/checklist-test/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@ node scripts/checklist-select.mjs <selector> --json
`blocked` 时才传 `--include-blocked`。**把解析器报出的 `revision` 钉进运行记录**:
判定只对它运行时所对的那个 revision 有效。

**`planned` 项任何选择器都不返回,也没有开关让它返回**:能力不存在,无 steps 可驱动、
无 oracle 可查,解析器单独列出命中项。运行记录逐项记 `planned`,⛔ 永不 pass / fail /
blocked / not-run,⛔ 不进标题计数;与 blocked 的分界是 fixture 债 vs 平台债。

## 1. 规划这一轮 —— 只构建需要的,钉住的先跑

读命中项的 `surface`:
Expand Down
63 changes: 62 additions & 1 deletion docs/qa/platform-checklist/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ next-sequential numbers do.
"id": "approvals.per-group-signoff", // "<area>.<slug>" — immutable, globally unique, never reused
"title": "Per-group sign-off (会签) needs one approval from EACH group",
"since": "v16", // release that introduced the capability
"status": "active", // active | draft | retired
"status": "active", // active | draft | planned | retired (see "Implementation status")
"revision": 1, // bumps on any semantic edit
"priority": "P1", // P0 = release-gating smoke · P1 = core · P2 = extended
"surface": "browser", // browser | api | cli | build | mixed (the 15.1 plan's 🖥/🔌 lanes)
Expand Down Expand Up @@ -297,6 +297,67 @@ Three things worth knowing before you meet it:
showcase-side fixture gaps #3358 uncovered (#3408, #3409, #3415) each cost a sweep to
rediscover; recording the gap on the item is what stops that.

## Implementation status — `planned` is how the ledger holds a capability gap

The ledger used to record only capabilities that already work, so a missing piece of a
listed capability had nowhere to live: the backlog sweep had nothing to point a `Path:`
at, and the platform's implementation status lived in nobody's head. The North Star's
definition line (「做出来的是什么」…「缺任何一样就不是这个应用」) makes such a piece a
**requirement**, so it belongs on the ledger — as a fourth status, not as a second
document. A separate feature list drifts against this one the moment either is edited,
and nothing can then say which of the two is wrong: there is no ratchet between two
hand-maintained documents, only a pair of disagreeing readings. Hence one ledger, one id
space, one more status — and a reading entry that carries numbers only because a command
produces them.

**`status: "planned"`** — the definition requires this capability and the platform does
not yet implement or verify it. What a planned item carries, and what it deliberately
does not:

| field | on a planned item |
|:---|:---|
| `id` · `title` · `priority` · `surface` · `revision` · `history` | as for any item — an id is picked once and is immutable, so it can be pointed at from the day the gap is found |
| `personas` | **required** — who the capability is for is knowable the day the gap is found, and is what makes the gap readable to the next sweep |
| `since` | `null` (no target release chosen) **or a TARGET release**. ⛔ Only the SHAPE is checked: this ledger holds no release timeline, so a target naming a release that already shipped is an authoring error no gate here can see |
| `steps` | **none.** There is nothing to drive. Steps arrive in the PR that implements the capability, in the same edit that promotes the item |
| `acceptance` | not required — no oracle can be consulted yet. Clauses drafted early are still validated |

Three consequences, all mechanical:

- **A planned item never runs.** `scripts/checklist-select.mjs` resolves it in no
selector's runnable set, and there is no flag that makes it one. The selector reports
it separately and the run record carries the verdict `planned` — ⛔ never `pass`,
`fail` or `blocked` ([RUNNER.md](./RUNNER.md) "Verdicts"). A blocked item is a real
test the environment cannot run today; a planned item has nothing to run at all.
- **A planned item is not coverage.** It is a legal `coverage.json` map target — that is
where a capability-gap card points — and it contributes zero: a kind whose *only*
items are planned is reported **UNMAPPED**, exactly as if the entry were empty.
Otherwise `planned` would be the cheapest way to green an untested kind, and the
ratchet would measure intentions instead of tests.
- **Promotion `planned → active` is earned, not declared.** It takes a run record in
which the item **passed**: steps and acceptance arrive with the implementation, the
item is run, and only then does the status flip (bumping `revision` and appending to
`history` like any other semantic edit). ⛔ Flipping the status because the code
landed is how a ledger starts reporting coverage it does not have.

**Reading the status** — one command, and the human entry point it publishes:

```bash
pnpm gen:checklist-status # per-area active/planned counts + every planned id
pnpm gen:checklist-status --out <dir> # also render the wiki pages into <dir>
```

The wiki index **[Platform-Checklist](https://github.com/objectstack-ai/objectstack/wiki/Platform-Checklist)**
is the 「平台功能清单 + 实现状态」 reading entry: one row per area with its active and
planned counts, linking to one `Checklist-<area>` page per area that lists every item
(`id · title · priority · status · personas`) with the planned ones in their own section
first. It is regenerated on a schedule by
[`.github/workflows/checklist-status.yml`](../../../.github/workflows/checklist-status.yml)
and is never committed here — ⛔ there is no `STATUS.md` in this tree and no gate paired
with one, by maintainer ruling: staleness is tolerated, and a generated page in the tree
is a third artifact to keep fresh whose stale copy reads exactly as authoritative as a
current one.

## Capability coverage — every capability the platform has gets tested

`coverage.json` makes "凡是有的能力, 都要测试" mechanical instead of aspirational. The
Expand Down
25 changes: 24 additions & 1 deletion docs/qa/platform-checklist/RUNNER.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,28 @@ Per **item**, derived — never hand-assigned:
- `partial` — some passed, none failed (the "proved half, left it unticked" state from
#3358, now first-class instead of a prose apology);
- `fail` — any clause failed;
- `blocked` / `not-run` — nothing consulted.
- `blocked` / `not-run` — nothing consulted;
- `planned` — the item's `status` is `planned`: the definition requires the capability
and the platform does not verify it yet. Read off the item, never derived from a run.

**No verdict without evidence.** A clause with no captured artifact is `not-run`, not
`pass`. Evidence means: the API/network trace, the screenshot, the log excerpt, or the
test-run output the clause's `evidence` field names.

**A `planned` item is never driven, and never scores.** `scripts/checklist-select.mjs`
returns it in no selector's runnable set — it prints the matches under a separate
PLANNED heading and hands the runner only what can run. Carry each one into the run
record as `planned`, with no clause table: there are no `steps` to follow and no
`acceptance` to consult, so ⛔ it is never `pass`, never `fail`, never `blocked` and
never `not-run`. The distinction that matters is against `blocked`: a blocked item is a
**real test** the environment cannot run today, so its gap is a fixture debt and the
right answer is to provision it; a planned item has **nothing to run at all**, so its
gap is a platform debt and the right answer is to build the capability. Scoring one as
the other sends the next sweep to fix the wrong thing. A planned item earns `active`
only through a run record in which it passed — see the README's "Implementation status".
The completion criterion is unchanged by them: an area is run when every item the
selector handed you has a verdict; planned ids are listed, not chased.

## The accuracy rules

1. **Oracle hierarchy** — server truth (`api`, `network`, `build`, `test`) outranks
Expand Down Expand Up @@ -429,6 +445,13 @@ QA run · <selector> (<judged>/<total>) · <framework-sha8> · <YYYY-MM-DD> · <
**An omitted bucket means "not declared", never zero** — the roll-up renders the two
differently, so write `0 FAIL` when you mean zero.

**⛔ `planned` items appear nowhere in this title** — not inside `<total>`, and not as a
sixth bucket. The counts vocabulary is closed at the five verdicts above, and a planned
item was never a judgeable unit of this run: putting one in `<total>` deflates the
coverage claim the record is read on, and inventing a `PLANNED` segment is a deviation
the roll-up refuses to parse. They are named in the body, under the scope section that
already records the selector.

**Retired phrasings — ⛔ none of these may be written again:** `(FULL area)`, `(N items)`,
`(N of M items)`, `(N/M items consulted)`, omitting the parenthetical entirely, and the
trailing `(11 not-run)` spelling of NOT-RUN. `(FULL area)` is why the ruling exists: it
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@
"check:scripts-symbol-anchors": "node scripts/symbol-anchors.mjs --self-test && node scripts/check-scripts-symbol-anchors.mjs --self-test && node scripts/check-scripts-symbol-anchors.mjs",
"check:spec-docblock-symbol-anchors": "node scripts/symbol-anchors.mjs --self-test && node scripts/check-spec-docblock-symbol-anchors.mjs --self-test && node scripts/check-spec-docblock-symbol-anchors.mjs",
"check:platform-checklist": "node scripts/checklist-select.mjs --self-test && node scripts/check-platform-checklist.mjs --self-test && node scripts/check-platform-checklist.mjs",
"gen:checklist-status": "node scripts/gen-checklist-status.mjs --self-test && node scripts/gen-checklist-status.mjs",
"check:org-identifier": "node scripts/check-org-identifier.mjs --self-test && node scripts/check-org-identifier.mjs",
"check:runner-env-posture": "node scripts/check-runner-env-posture.mjs --self-test && node scripts/check-runner-env-posture.mjs",
"check:cli-test-child-env": "node scripts/check-cli-test-child-env.mjs --self-test && node scripts/check-cli-test-child-env.mjs",
Expand Down
Loading
Loading