fix: derive npm publish set instead of hardcoding package list by dmealing · Pull Request #308 · metaobjectsdev/metaobjects

dmealing · 2026-08-18T02:48:11Z

Intent

Fix .github/workflows/publish-npm.yml, which would publish an uninstallable @metaobjectsdev/cli.

THE BUG. The workflow publishes a hardcoded list of 13 package directories. server/typescript/packages/docs-site is not in that list, but @metaobjectsdev/docs-site (a) is not private so it rides the lockstep version like every other published package, and (b) is a runtime 'dependencies' entry of @metaobjectsdev/cli (workspace:*, rewritten to the concrete version at pack time). scripts/release.mjs derives its publish set dynamically (every non-private package whose version equals the CLI's), so the LOCAL release path publishes all 14 and docs-site@0.23.2 is on npm today. A release cut through the WORKFLOW instead would publish a cli whose docs-site dependency does not exist at that version - an uninstallable CLI (npm ETARGET), discoverable only by an external npm install. Two answers to one question that had drifted, and nothing had ever compared them.

WHAT WAS ASKED FOR, and the decisions made:

Make the workflow's publish set correct. The task PREFERRED replacing the hardcoded list with the same derivation scripts/release.mjs already uses so the two paths cannot drift again, over the fallback of adding docs-site to the list plus a divergence check. I took the preferred option: new scripts/publish-set.mjs is the single source of truth, both publish paths read it, and the workflow's list is DELETED rather than corrected - a list that no longer exists cannot drift from the derivation. release.mjs now imports publishSet() instead of enumerating and sorting itself.
Fix the tier ordering. TIER_ORDER omitted docs-site, so indexOf returned -1 and it sorted FIRST. Verified this is worse than the task described: it is not merely 'harmless because nothing but cli depends on it' - sorting first put docs-site ahead of metadata and render, the two packages it DOES depend on, so the local path was already violating the documented tier contract. docs-site now has a declared tier (after sdk, before runtime-web - matching docs/RELEASING.md's tier table and scripts/prerelease.mjs's TIERS, both of which were already correct), and an undeclared tier is now a hard error rather than an accidental position.
Prove the failure and the fix, not just reason about it. Packed the cli tarball and showed it pins @metaobjectsdev/docs-site@0.23.2; extracted the pre-change workflow's list and showed docs-site absent (13 vs the derived 14); showed the derived list after the change is identical to the derivation and closed over every packed-cli sibling dep; and showed the guard now throws on the old TIER_ORDER. Evidence goes in the PR body.

DELIBERATE DESIGN CHOICES a reviewer might otherwise question:

publish-set.mjs throws on THREE invariants, not just the one that caused the bug: an untiered member, a set not closed over its own sibling runtime deps (the defect itself), and a tier order that would publish a dependency after its dependent. It throws rather than returning a wrong set because every caller is about to do something irreversible (npm versions are permanent).
Offline by construction - it reads manifests, never the registry, matching the existing scripts/check-publish-intent.sh precedent ('what SHOULD we publish' is a fact about this tree, not about npm).
publishSet() takes an options object with overridable root/tierOrder purely so scripts/test-publish-set.mjs can drive each invariant against a synthetic throwaway workspace; production callers pass nothing. The real tree is by design always valid, so it can only ever prove the happy path - hence synthetic fixtures for the negative cases, plus one regression test asserting docs-site is in the set and ordered before cli in the REAL tree.
Wired into scripts/ci-local.sh's gates lane as 'publish-set parity', beside check-publish-intent.sh which enforces the same rule from the other side. Verified: scripts/ci-local.sh --only gates is 10/10 green.
Drive-by in the step I was already rewriting: added a trap to remove ~/.npmrc on failure (previously only removed on success), and the derived list is written to $RUNNER_TEMP rather than the repo working dir. The list is read via mapfile from a file (not a pipe) so a non-zero exit from the derivation fails the step instead of silently yielding a short list, and so bun publish cannot consume the list from stdin.

SCOPE CONSTRAINTS the user set, and how they were honored:

Public repository: no home paths, personal identifiers, private hostnames or credentials in any committed file. Verified - staged diff greps clean for /home/, /Users/, scratchpad and firstmate refs, and .githooks/leak-scan.sh reports clean.
Nothing was published to any registry, public or private. This is release plumbing only. The publish loop was verified with bun publish stubbed out to an echo.
Scope limited to the publish-set correctness bug and the tier ordering. In particular scripts/prerelease.mjs was NOT touched even though it carries a third copy of the tier list - the user explicitly said not to touch the pre-release tooling that just landed. That copy already contains docs-site and is correct today; folding it onto the shared module is called out in the PR body as an obvious follow-up rather than done here. release.mjs was not refactored beyond what the fix needed.

ALSO IN THIS COMMIT, and not part of the fix: per the managing harness's project-memory step, CLAUDE.md was promoted to AGENTS.md with CLAUDE.md kept as a tracked symlink (git mode 120000), so every existing reference to CLAUDE.md across README/CONTRIBUTING/port docs still resolves. The durable rule ('never hardcode the npm publish set') was recorded in AGENTS.md beside the existing Publishing pointer. This rename is a repo-convention change riding along in a release-plumbing PR; it was flagged to the user as separable. Also added a CHANGELOG [Unreleased] entry under the existing prerelease/release-fix sections.

Pre-existing and NOT introduced here: two SC2164 shellcheck warnings in scripts/ci-local.sh (confirmed present at HEAD before this change, count unchanged); the new workflow step body is shellcheck-clean.

What Changed

Replaced hardcoded 13-package list in .github/workflows/publish-npm.yml with dynamic derivation via new scripts/publish-set.mjs (single source of truth that validates dependency closure and topological tier ordering)
Added docs-site to TIER_ORDER and made untiered packages throw instead of silently sorting first (which put them ahead of their own dependencies)
Added scripts/test-publish-set.mjs test suite and CI gate check in scripts/ci-local.sh ('publish-set parity')

Risk Assessment

✅ Low: Well-tested fix with comprehensive behavioral tests, proper error handling, and good safeguards against future drift; the one finding is a minor test-quality issue that doesn't affect correctness.

Testing

Verified the publish-set fix through unit tests (6 invariant checks including regression test), derivation output (14 packages with docs-site at position #7, cli at #13), CLI tarball inspection (docs-site@0.23.2 present in packed dependencies), and CI gates integration (publish-set parity gate passes). All tests confirm the bug is fixed, the root cause eliminated, and future drift prevented.

Evidence: Test Summary

# Publish Set Fix - Test Evidence

## Bug Description
The `.github/workflows/publish-npm.yml` workflow had a hardcoded list of 13 package 
directories to publish, while `scripts/release.mjs` derived its publish set dynamically. 
The lists drifted: `@metaobjectsdev/docs-site` is a runtime dependency of 
`@metaobjectsdev/cli` but was missing from the workflow's list. A release cut through 
the workflow would have published a CLI tarball pinning `docs-site@0.23.2` when that 
version was never published to npm → ETARGET, uninstallable CLI.

## Fix Summary
Created `scripts/publish-set.mjs` as the single source of truth for WHICH packages to 
publish and in WHAT ORDER. Both `scripts/release.mjs` and the workflow now read from it. 
The workflow's hardcoded list was deleted, not just corrected.

## Test Results

### 1. Unit Tests (`scripts/test-publish-set.mjs`)
All 6 test cases pass:
- ✓ Happy path: derives correct set in tier order
- ✓ Throws when package has no declared tier (prevents docs-site bug recurrence)
- ✓ Throws when set not closed over runtime deps (the bug itself)
- ✓ Throws when tier order would publish a dep after its dependent
- ✓ Regression: docs-site is in the set
- ✓ Regression: docs-site publishes before cli

### 2. Derivation Verification
Running `node scripts/publish-set.mjs --check` on the real tree:
- ✓ Produces 14 packages (not 13)
- ✓ docs-site is at position #7
- ✓ cli is at position #13 (after docs-site)
- ✓ All invariants satisfied: tier-ordered, closed over sibling deps

### 3. CLI Tarball Verification
Packed the CLI tarball with `bun pm pack`:
- ✓ All workspace:* dependencies rewritten to concrete versions
- ✓ @metaobjectsdev/docs-site: 0.23.2 is present in dependencies
- ✓ All 9 @metaobjectsdev/* deps pinned to 0.23.2

### 4. Workflow Integration
The workflow now:
- ✓ Runs `node scripts/publish-set.mjs --check` before publish
- ✓ Derives the directory list dynamically (no hardcoded list)
- ✓ Fails explicitly on empty set or non-zero derivation exit
- ✓ Includes docs-site in the publish loop

### 5. CI Gate Integration
The `gates` lane in `scripts/ci-local.sh`:
- ✓ New gate "publish-set parity" added
- ✓ Runs derivation + all tests
- ✓ All 10 gates pass

## Before/After Comparison

OLD: 13 hardcoded directories in workflow (missing docs-site)
NEW: 14 derived directories (includes docs-site)

The derivation enforces three invariants:
1. Every package has a declared tier
2. Set is closed over its runtime dependencies
3. Tier order respects dependency order

## Conclusion
The fix is complete and verified:
- ✓ The bug (missing docs-site) is fixed
- ✓ The root cause (two answers to one question) is eliminated
- ✓ Future drift is prevented (single source of truth)
- ✓ All invariants are tested and enforced
- ✓ CI gates verify correctness on every commit

Evidence: Publish Set Derivation

$ node scripts/publish-set.mjs --check

   1. @metaobjectsdev/metadata  (server/typescript/packages/metadata)
   2. @metaobjectsdev/render  (server/typescript/packages/render)
   3. @metaobjectsdev/codegen-ts  (server/typescript/packages/codegen-ts)
   4. @metaobjectsdev/runtime-ts  (server/typescript/packages/runtime-ts)
   5. @metaobjectsdev/migrate-ts  (server/typescript/packages/migrate-ts)
   6. @metaobjectsdev/sdk  (server/typescript/packages/sdk)
   7. @metaobjectsdev/docs-site  (server/typescript/packages/docs-site)
   8. @metaobjectsdev/runtime-web  (client/web/packages/runtime-web)
   9. @metaobjectsdev/codegen-ts-react  (server/typescript/packages/codegen-ts-react)
  10. @metaobjectsdev/codegen-ts-tanstack  (server/typescript/packages/codegen-ts-tanstack)
  11. @metaobjectsdev/react  (client/web/packages/react)
  12. @metaobjectsdev/tanstack  (client/web/packages/tanstack)
  13. @metaobjectsdev/cli  (server/typescript/packages/cli)
  14. @metaobjectsdev/ai-runtime  (server/typescript/packages/ai-runtime)
publish set: OK (lockstep 0.23.2; 14 packages, tier-ordered, closed over sibling deps)

KEY FINDINGS:
✓ 14 packages (not 13 as the old hardcoded list had)
✓ docs-site is at position #7
✓ cli is at position #13, AFTER docs-site
✓ All invariants satisfied: tier-ordered, closed over sibling deps

Evidence: CLI Tarball Dependencies

$ bun pm pack --destination /tmp/test-pack --cwd server/typescript/packages/cli
[...packing files...]

$ tar -xzOf /tmp/test-pack/*.tgz package/package.json | jq -r '.dependencies | to_entries | map(select(.key | startswith("@metaobjectsdev/"))) | .[] | "\(.key): \(.value)"'

@metaobjectsdev/codegen-ts: 0.23.2
@metaobjectsdev/codegen-ts-react: 0.23.2
@metaobjectsdev/codegen-ts-tanstack: 0.23.2
@metaobjectsdev/docs-site: 0.23.2
@metaobjectsdev/metadata: 0.23.2
@metaobjectsdev/migrate-ts: 0.23.2
@metaobjectsdev/render: 0.23.2
@metaobjectsdev/runtime-ts: 0.23.2
@metaobjectsdev/sdk: 0.23.2

KEY FINDINGS:
✓ All workspace:* dependencies were rewritten to concrete versions
✓ @metaobjectsdev/docs-site: 0.23.2 is present (would be ETARGET if not published)
✓ All @metaobjectsdev/* deps are pinned to the same version (0.23.2)

Evidence: Publish Set Tests

$ node scripts/test-publish-set.mjs

ok:   happy path: private + off-lockstep excluded, order [metadata, render, docs-site, cli]
ok:   untiered member: threw on "publish order undeclared"
ok:   unclosed set: threw on "not closed over its own runtime dependencies"
ok:   inverted tier order: threw on "publishes a dependency AFTER its dependent"
ok:   regression: docs-site (#6) publishes before cli (#12) in the real tree
ok:   regression: docs-site has a declared tier

publish-set tests: all passed

KEY FINDINGS:
✓ Happy path: derives correct set in tier order
✓ Invariant 1: throws when a package has no declared tier
✓ Invariant 2: throws when set not closed over runtime deps (the bug itself)
✓ Invariant 3: throws when tier order would publish a dep after its dependent
✓ Regression test: docs-site is in the set and publishes before cli

Evidence: Workflow Comparison

OLD WORKFLOW (hardcoded list of 13 directories):
  server/typescript/packages/metadata
  server/typescript/packages/render
  server/typescript/packages/codegen-ts
  server/typescript/packages/runtime-ts
  server/typescript/packages/migrate-ts
  server/typescript/packages/sdk
  client/web/packages/runtime-web
  server/typescript/packages/codegen-ts-react
  server/typescript/packages/codegen-ts-tanstack
  client/web/packages/react
  client/web/packages/tanstack
  server/typescript/packages/cli
  server/typescript/packages/ai-runtime

MISSING: server/typescript/packages/docs-site ⚠️

NEW WORKFLOW (derived from scripts/publish-set.mjs):
server/typescript/packages/metadata
server/typescript/packages/render
server/typescript/packages/codegen-ts
server/typescript/packages/runtime-ts
server/typescript/packages/migrate-ts
server/typescript/packages/sdk
server/typescript/packages/docs-site
client/web/packages/runtime-web
server/typescript/packages/codegen-ts-react
server/typescript/packages/codegen-ts-tanstack
client/web/packages/react
client/web/packages/tanstack
server/typescript/packages/cli
server/typescript/packages/ai-runtime

KEY FINDINGS:
✓ Old workflow: 13 packages (hardcoded list)
✓ New workflow: 14 packages (derived, including docs-site)
✓ The lists can never drift again - single source of truth
✓ Workflow now runs `node scripts/publish-set.mjs` to get the list dynamically

Evidence: Workflow Integration

WORKFLOW INTEGRATION (.github/workflows/publish-npm.yml):

NEW STEPS ADDED:

1. Derive + verify the publish set:
   - name: Derive + verify the publish set
     run: node scripts/publish-set.mjs --check

2. Publish loop now uses derived set:
   OLD (hardcoded):
     for d in server/typescript/packages/metadata server/typescript/packages/render \
              server/typescript/packages/codegen-ts server/typescript/packages/runtime-ts \
              ... (13 total)
   
   NEW (derived):
     node scripts/publish-set.mjs > "${RUNNER_TEMP}/publish-set.txt"
     mapfile -t DIRS < "${RUNNER_TEMP}/publish-set.txt"
     [ "${#DIRS[@]}" -gt 0 ] || { echo "::error::derived publish set is empty"; exit 1; }
     echo "publishing ${#DIRS[@]} packages in tier order"
     for d in "${DIRS[@]}"; do
       pub "$d"
     done

SAFEGUARDS:
✓ Derivation runs with --check before publish (catches invariant violations)
✓ Non-zero exit from derivation fails the workflow step
✓ Empty set is detected and fails explicitly
✓ ~/.npmrc cleanup now happens on both success and failure (via trap)

CI GATE INTEGRATION (scripts/ci-local.sh):
✓ New gate: "publish-set parity"
✓ Runs: node scripts/publish-set.mjs --check && node scripts/test-publish-set.mjs
✓ Verifies derivation + runs all tests

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info

ℹ️ scripts/test-publish-set.mjs:115 - Redundant source-content check: TIER_ORDER.includes("docs-site") reads the TIER_ORDER constant directly rather than testing behavior. The behavioral invariant (publishSet() throws if a member is untiered) is already tested by the publishSet() call on line 105, which would have thrown if docs-site were not in TIER_ORDER. This check tests source content rather than executable behavior.

✅ **Test** - passed

✅ No issues found.

node scripts/test-publish-set.mjs - all 6 unit tests pass
node scripts/publish-set.mjs --check - derivation produces 14 packages, tier-ordered, closed over deps
bun pm pack CLI tarball - verified docs-site@0.23.2 is present in packed dependencies
scripts/ci-local.sh --only gates - all 10 gates pass including new publish-set parity gate

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

`.github/workflows/publish-npm.yml` carried its own hardcoded list of 13 package directories while `scripts/release.mjs` DERIVED the same set (every non-private package at the CLI's version). Two answers to one question, and they had drifted: `@metaobjectsdev/docs-site` is a runtime `dependencies` entry of `@metaobjectsdev/cli` and was not in the workflow's list, so a release cut through the workflow would have published a `cli` pinning a `docs-site` version nobody published — `npm i @metaobjectsdev/cli` → ETARGET. Only an external install could have found it; the local release path publishes all 14, which is what kept it latent (docs-site@0.23.2 is on npm today). `scripts/publish-set.mjs` is now the single source of truth for which packages ship and in what order. Both publish paths read it and the workflow's list is gone, so it cannot drift from a derivation it no longer has. The derivation throws rather than returning a wrong set: a member with no declared tier, a set not closed over its own sibling runtime deps, or a tier order that would publish a dependency after its dependent. TIER_ORDER omitted `docs-site` too, and that was not harmless: `indexOf()` returns -1, which does not sort last — it sorts FIRST, so the local path published docs-site ahead of `metadata` and `render`, the two packages it depends on. The tier is declared now, and an undeclared one is an error. - scripts/publish-set.mjs — the shared derivation + its three invariants - scripts/test-publish-set.mjs — each invariant driven against a synthetic workspace, plus a regression pinning docs-site before cli in the real tree - scripts/ci-local.sh — new `publish-set parity` gate in the gates lane - docs/RELEASING.md, AGENTS.md, CHANGELOG.md — record the rule Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

dmealing and others added 2 commits August 17, 2026 22:30

no-mistakes(document): Fixed tier table order in docs/RELEASING.md

e8849c5

dmealing merged commit 79bedda into main Aug 18, 2026
1 check passed

dmealing deleted the fm/mo-docs-site-pub-j8 branch August 18, 2026 09:47

Navigation Menu

Uh oh!

fix: derive npm publish set instead of hardcoding package list - #308

fix: derive npm publish set instead of hardcoding package list#308
dmealing merged 2 commits into
mainfrom
fm/mo-docs-site-pub-j8

dmealing commented Aug 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

dmealing commented Aug 18, 2026

Intent

What Changed

Risk Assessment

Testing

Pipeline

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant