diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 71cc6713a..3fe220ff2 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -43,6 +43,21 @@ hits=$(git diff --cached --diff-filter=ACM -U0 --no-color 2>/dev/null | awk ' /^\+/ && !/^\+\+\+/ { if (f !~ /^\.githooks\//) print f "\t" substr($0,2) } ' | grep -Ei "$PATTERNS" | grep -vEi "$ALLOW") +# A pre-release version must never be committed: scripts/release.mjs derives the lockstep +# set from the CLI's CURRENT version, so a stray -rc.N silently shrinks the next release. +# Only runs when a version-bearing file is actually staged. +if git diff --cached --name-only --diff-filter=ACM 2>/dev/null | grep -qE \ + '(package\.json|pom\.xml|pyproject\.toml|Directory\.Build\.props)$'; then + if [ -x "$ROOT/scripts/check-no-prerelease-versions.sh" ]; then + if ! "$ROOT/scripts/check-no-prerelease-versions.sh" >/dev/null 2>&1; then + echo "" >&2 + "$ROOT/scripts/check-no-prerelease-versions.sh" >&2 || true + echo " commit blocked — see above. Bypass (discouraged): git commit --no-verify" >&2 + exit 1 + fi + fi +fi + if [ -n "$hits" ]; then echo "" >&2 echo " commit blocked — possible private/other-project or local-path leak (metaobjects is PUBLIC):" >&2 diff --git a/.gitignore b/.gitignore index c4111f6b1..8f4ff85a1 100644 --- a/.gitignore +++ b/.gitignore @@ -66,3 +66,7 @@ server/java/.claude/ # Serena MCP project cache (local tooling scratch) .serena/ + +# Private pre-release registry address + token (docs/features/prerelease.md). +# Never committed: this repository is PUBLIC. +tools/prerelease/registry.env diff --git a/CHANGELOG.md b/CHANGELOG.md index d93484ccb..1ae9f4b17 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,64 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ## [Unreleased] +### Added — pre-release publishing to a private registry (no more real releases just to test a change) + +Trying an unreleased change against a downstream project required cutting a real release on +npm / PyPI / NuGet / Maven Central. All four are immutable, so every experiment spent a +version number, moved `latest`, and was visible to every consumer on a caret range. There is +now a private path: publish a **pre-release** to a separate registry, consume it downstream, +iterate, and switch back with one verified command. + +- **`bun run prerelease`** (`scripts/prerelease.mjs`) — publishes the in-development version + to a registry configured in `tools/prerelease/registry.env` (gitignored). One canonical + version string `-rc.`, normalized in exactly one place: `0.24.0-rc.3` (npm, + NuGet), `0.24.0rc3` (PEP 440), `7.24.0-rc.3` (Maven). npm by default, `--only all` for the + four ports. The collision-breaker is a **counter, not a commit sha**, because npm strips + SemVer build metadata — `0.24.0-rc.1+aaa` and `+bbb` are the same version to it. +- **`tools/prerelease/prerelease-link.sh link|unlink|check`** — points a downstream project + at the registry, and takes it back off. It detects the project's ecosystems, writes only + namespace-scoped config (`@metaobjectsdev/*`, `metaobjects`, `MetaObjects*`, + `com.metaobjects` — everything else keeps resolving publicly), and on `unlink` repins + **every** vendor dependency, drops the lockfile, and runs the detector to prove the + project is clean. Repinning only the dependency you installed is not enough: `meta init` + writes `@metaobjectsdev/codegen-ts` and `@metaobjectsdev/metadata` into a consumer's + devDependencies too, and missing them fails the next clean install with `notarget`. +- **`tools/prerelease/detect-prerelease-pins.sh`** — the guard a consumer commits and runs + in CI. The registry is a public HTTPS endpoint with anonymous reads, so no network + boundary is doing safety work; this check *is* the containment. It scans dependency + declarations only (a test server bound to `127.0.0.1` is not a dependency on anything) and + knows the registry host by default, so a consumer repo that has never seen the publisher's + config still catches a leak. +- **`scripts/check-no-prerelease-versions.sh`** — wired into `.githooks/pre-commit` and the + `gates` lane. A committed `-rc.N` is not cosmetic: `scripts/release.mjs` derives the + lockstep set from the CLI's *current* version, so one stray pre-release version silently + drops that package from the next real release. +- `tools/prerelease/docker-compose.yml` + `bootstrap.sh` stand up an equivalent registry for + a fork or an offline machine; the publisher is registry-agnostic either way. +- Adopter-facing guide: [`docs/features/prerelease.md`](docs/features/prerelease.md). + +**Config is per-project and never machine-global**, deliberately. A user-level `~/.npmrc` is +invisible to the detector, switches every project at once, and — the reason this is a rule +rather than a preference — a silent fall-back to user-level config is the exact mechanism +that published a pre-release to public npm while this was being built: `bun publish` ignores +`npm_config_userconfig`, found `~/.npmrc`, and shipped for real. Every publish path now +asserts its target equals the configured registry, checks it against a deny-list of the +public registries, **parses `bun publish --dry-run`** rather than trusting bun, and runs with +`HOME` redirected so a fall-back has no credential to use. + +### Fixed — `scripts/release.mjs` preflighted only one package + +The target-version check ran `npm view @metaobjectsdev/cli@` and nothing else, so a +version already published for any *other* package in the lockstep set was discovered +mid-publish — after its dependencies had shipped irreversibly. That is not hypothetical: +`@metaobjectsdev/metadata@0.24.0-rc.1` exists on public npm and no other package in the set +carries it, so a lockstep RC at `0.24.0-rc.1` would publish thirteen packages and then fail +on the fourteenth. npm versions cannot be reclaimed — `unpublish` is *refused* (`E405`) once +anything depends on the version, and deprecation does not free the number. The preflight now +checks every package in the set (in parallel, so it stays fast), and `bun run prerelease` +skips numbers already burned on public npm when choosing an iteration. + + ## [0.23.2] — npm `0.23.2` · PyPI `0.23.2` · NuGet `0.23.2` · Maven `7.23.2` A coordinated **PATCH** across all four registries. diff --git a/CLAUDE.md b/CLAUDE.md index 56b06e664..99ee983f2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -565,7 +565,7 @@ PRs welcome. When contributing: For significant new features or architectural changes, open an issue first to discuss the approach. -**Publishing to npm:** see [docs/RELEASING.md](docs/RELEASING.md) — the procedure (RC → smoke-test → promote) plus the non-obvious gotchas (publish with `bun`, regen the lockfile after every version bump, runtime imports must be `dependencies`, verify a real external install in npm *and* pnpm). +**Publishing:** To iterate an unreleased change against a downstream project, use [docs/features/prerelease.md](docs/features/prerelease.md) — publish to the private registry (`bun run prerelease`), consume it, iterate, and revert with one verified command. For full public releases, see [docs/RELEASING.md](docs/RELEASING.md) — the procedure (RC → smoke-test → promote) plus the non-obvious gotchas (publish with `bun`, regen the lockfile after every version bump, runtime imports must be `dependencies`, verify a real external install in npm *and* pnpm). ## Roadmap pointer diff --git a/docs/RELEASING.md b/docs/RELEASING.md index 86de16f7e..5498db19d 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -112,8 +112,14 @@ Publish in tier order so a dependent never lands before its dependency. **`forge pnpm** (pnpm's strict, non-nested `node_modules` exposes resolution bugs npm/bun hide). Install the cli into a throwaway dir, run `meta --version`, `meta init`, `meta gen`. -5. **npm versions are immutable.** You can never re-publish a version, and unpublish is a - restricted 72-hour escape hatch. That's why we go RC-first. +5. **npm versions are immutable, and a burned one never comes back.** You can never + re-publish a version. Unpublish is not a reliable escape hatch: it is *refused* (`E405`) + once anything depends on the version, and deprecating it does not free the number. + `@metaobjectsdev/metadata@0.24.0-rc.1` is burned that way and no other package in the set + carries it — so a lockstep RC at `0.24.0-rc.1` would publish thirteen packages and then + fail irreversibly on the fourteenth. `scripts/release.mjs` now preflights the target + version against **every** package in the set (it used to check only the cli), and + `bun run prerelease` skips burned numbers when choosing an iteration. ## Versioning policy (pre-1.0) @@ -224,6 +230,20 @@ bun run clean && bun run build Spot-check `dist` reflects the change (a deleted source's `.js` is gone, new code present). ### 1. Release candidate → `next` + +> **Most changes do not need this.** To try an unreleased change against a downstream +> project, publish a PRE-RELEASE to the private registry instead — +> [`docs/features/prerelease.md`](features/prerelease.md), `bun run prerelease`. It is +> reversible, invisible to the public registries, and costs no version number. +> +> A **public** RC is for the one case a private registry cannot cover: **dependencies or +> package layout changed**, so the thing being tested IS a real external install from the +> real registry — a misclassified `dependencies`/`devDependencies` entry, a peer range, a +> new package name, an `exports` map. Rule 4 below only means something against npmjs.org. +> +> Remember what it costs: an RC version is permanent. Once anything depends on it, +> `npm unpublish` is refused outright and deprecating it does not free the number. + ```bash # bump the candidate set to -rc.N (sed the "version" field in each publish-candidate package.json) rm bun.lock && bun install # CRITICAL — re-pins workspace versions diff --git a/docs/features/prerelease.md b/docs/features/prerelease.md new file mode 100644 index 000000000..eb6ad2d28 --- /dev/null +++ b/docs/features/prerelease.md @@ -0,0 +1,290 @@ +# Pre-releases: iterating an unreleased version against downstream consumers + +Testing an unreleased change against a real downstream project used to require cutting a +real release on npm / PyPI / NuGet / Maven Central. All four registries are **immutable**: +the version number is spent the moment it ships, `latest` moves, and every consumer on a +caret range can pick it up. That is an expensive way to answer "does this change work in a +real app?" — and it is the reason `0.21.2` had to be cut within an hour of `0.21.1`. + +This page describes the alternative. Publish a **pre-release** to a **separate registry**, +consume it from a downstream project, iterate, and switch that project back to public +releases with a verified one-command revert. + +| | | +|---|---| +| Registry | `https://gitea.mealing.com` — one Gitea instance serving npm, PyPI, NuGet and Maven | +| Reads | **anonymous** — a consumer needs the URL and the owner, no account and no token | +| Writes | token only, in gitignored local config, never in a committed file | +| Publisher | `bun run prerelease` (`scripts/prerelease.mjs`) | +| Consumer | `tools/prerelease/prerelease-link.sh link` / `unlink` | +| Guard (consumer) | `tools/prerelease/detect-prerelease-pins.sh` | +| Guard (this repo) | `scripts/check-no-prerelease-versions.sh` | + +--- + +## 1. The version scheme + +One canonical internal string, normalized per ecosystem in exactly one place +(`const V` in `scripts/prerelease.mjs`): + +| | canonical | npm | PyPI | NuGet | Maven | +|---|---|---|---|---|---| +| form | `-rc.` | `0.24.0-rc.3` | `0.24.0rc3` | `0.24.0-rc.3` | `7.24.0-rc.3` | +| why | | SemVer2 verbatim | PEP 440 canonical form | SemVer2 verbatim | same `minor.patch` on the historical major `7` | + +`` is the in-development version — the next minor by default — and `` is a +**monotonic iteration counter**, derived from what the registry already holds across all +four ecosystems so that `--only npm` today and `--only csharp` tomorrow cannot collide. + +**Why a counter and not a commit sha.** npm *strips* SemVer build metadata: +`0.24.0-rc.1+aaa` and `0.24.0-rc.1+bbb` compare **equal**, so the second publish is refused +as a duplicate. The sha still travels where it is useful — the C# packages carry it in +`AssemblyInformationalVersion` via Source Link — but it cannot be the thing that makes two +iterations distinct. + +**Why not one mutable `-dev` version.** Deleting and re-pushing the same version is +possible, and it silently serves the consumer **stale bytes**: with a lockfile and a warm +client cache, `npm install` resolves the old tarball with no error and no warning. An +immutable per-iteration version makes "did my fix actually reach the consumer?" answerable +by reading a version number. + +`-rc.N` sorts correctly everywhere, including numerically in Maven (`rc.2` before `rc.10`), +and it sorts **below** the eventual release. Neither `^0.23.2` nor `^0.24.0` matches +`0.24.0-rc.1`, so a pre-release can never be picked up by an existing range. + +### A burned version number can never come back + +npm versions are permanent in a stronger sense than "you should not republish": once +anything depends on a version, `npm unpublish` is **refused** (`E405`), and deprecating it +does not free the number. `@metaobjectsdev/metadata@0.24.0-rc.1` is burned exactly that +way — published to public npm by accident while this design was being validated, and now +unremovable. Nothing else in the lockstep set carries it, which is what makes it dangerous: +a lockstep RC at `0.24.0-rc.1` would publish thirteen packages successfully and then fail +irreversibly on the fourteenth. + +Two places now handle that instead of discovering it late: + +- `bun run prerelease` picks its iteration number by skipping every number already taken on + the pre-release registry **or on public npm**, for any package in the set. With + `0.24.0-rc.1` burned and `rc.1`–`rc.3` used privately, it selects `rc.4`. An explicit + `--iter` that lands on a burned number still works — the pre-release registry is a + separate namespace — but warns that the number can never be promoted. +- `bun run release` checks the target version against **every** package in the lockstep + set before it publishes anything. It previously checked only `@metaobjectsdev/cli`, which + would not have seen this at all. + +> Do **not** use `-next.N`. Maven treats `next` as an *unknown* qualifier, which ranks +> **above** the plain release: `7.24.0-next.3` sorts newer than `7.24.0`. + +--- + +## 2. Publishing + +One-time, on the publishing machine: + +```bash +cp tools/prerelease/registry.env.example tools/prerelease/registry.env +# fill in MO_REGISTRY_OWNER and MO_REGISTRY_TOKEN — the file is gitignored +``` + +Then: + +```bash +bun run prerelease # next iteration, npm (the default scope) +bun run prerelease --only python,csharp # pick ports +bun run prerelease --only all # all four +bun run prerelease --iter 7 # pin the iteration number +bun run prerelease --base 0.25.0 # target a different in-development version +bun run prerelease --dry-run # build + normalize + gate, publish nothing +``` + +Version declarations are edited in place and **always restored on exit**; the script +refuses to start if any of them is already dirty. + +### The publish-target gate + +The registry is a public HTTPS endpoint. "It is only bound to loopback" is not the safety +model and never was the durable one. These are: + +1. The target must equal the **configured** registry (or be loopback) — an equality test, + not a hostname pattern. +2. An independent deny-list of the public registries (`registry.npmjs.org`, `pypi.org`, + `api.nuget.org`, `central.sonatype.com`, …). Two checks that fail differently beat one + check trusted twice. +3. For npm, `bun publish --dry-run` is **parsed** and its reported registry compared to the + expected one. This is not paranoia: bun ignores `npm_config_userconfig`, and during this + design's validation it silently fell back to the user-level `~/.npmrc` and published a + pre-release to the **public** registry. bun is not taken at its word anywhere here. +4. `HOME` is redirected to a scratch directory holding only the pre-release `.npmrc`, so a + fall-back has no credential to publish with even if it happens. +5. Maven deploys with an explicit `-DaltDeploymentRepository` and never `-Prelease` — this + repo declares `distributionManagement` only inside the `release` profile, so a bare + `mvn deploy` has no target at all. Its local repository is a scratch directory, so a + pre-release never lands in the `~/.m2` that ordinary builds resolve from. + +--- + +## 3. Consuming — one command in each direction + +```bash +# from the consumer project root +tools/prerelease/prerelease-link.sh link --version 0.24.0-rc.3 +tools/prerelease/prerelease-link.sh check +tools/prerelease/prerelease-link.sh unlink --to 0.23.2 +``` + +`link` detects which ecosystems the project uses and configures only those, scoped to the +vendor namespaces (`@metaobjectsdev/*`, `metaobjects`, `MetaObjects*`, `com.metaobjects`). +Everything else keeps resolving from the public registry — verified: `zod` and `pyyaml` +still come from npmjs.org and pypi.org while the vendor packages come from the pre-release +registry. + +Everything it writes is delimited by managed markers, so `unlink` removes exactly what +`link` added: + +| ecosystem | what `link` writes | mechanism | +|---|---|---| +| npm | `.npmrc` scope line | `@metaobjectsdev:registry=…` | +| Python | `pyproject.toml` block | `[[tool.uv.index]] explicit = true` + `[tool.uv.sources]` — real per-package index pinning | +| NuGet | `NuGet.config` | a second source plus `packageSourceMapping` limiting it to `MetaObjects*` | +| Maven | `pom.xml` `` | plus `.mvn/settings.xml` when the registry is plain `http` (see below) | + +Files that are not normally tracked (`.npmrc`, `NuGet.config`, `.mvn/*`) are also added to +the project's `.git/info/exclude`, which is local and not committed. Files that *are* +tracked by definition (`pyproject.toml`, `pom.xml`) get a loud warning instead — there is +no way to make an edit to a tracked file uncommittable, which is precisely why the detector +in §5 exists. + +After `link`: + +```bash +npm rm -f package-lock.json && npm install +python uv lock && uv sync +nuget dotnet restore --force-evaluate --no-cache +maven mvn -U compile +``` + +> **NuGet's two flags are both required.** NuGet caches the service index, so a plain +> `dotnet restore` — and even `--force-evaluate` on its own — will happily keep resolving +> the previous iteration of a floating version. + +> **Maven blocks plain-http repositories** since 3.8.1, via a built-in +> `maven-default-http-blocker` mirror, and the error names the blocker rather than the +> cause. Against an `http://` registry `link` writes `.mvn/settings.xml` + `.mvn/maven.config` +> using `-gs`, which **merges** with your own `~/.m2/settings.xml` rather than replacing it. +> `unlink` deletes both — leaving them behind would keep a security default suspended for +> that project forever. The project registry is HTTPS, so this path does not trigger for it. + +### Outside collaborators + +Reads are anonymous, so someone outside the project needs no account and no token. Give +them the two scripts (or a checkout of this repo) and one variable: + +```bash +MO_REGISTRY_OWNER= tools/prerelease/prerelease-link.sh link --version 0.24.0-rc.3 +npm install +``` + +and to get back off it: + +```bash +tools/prerelease/prerelease-link.sh unlink # --to defaults to the current npm `latest` +npm install +``` + +The registry host is defaulted in the tooling, so `MO_REGISTRY_OWNER` is the only thing +they need to be told. `unlink` needs nothing at all. + +--- + +## 4. Why the config is per-project and never machine-global + +A user-level `~/.npmrc`, `~/.m2/settings.xml`, `~/.config/NuGet/NuGet.Config` or `~/.pypirc` +would be less typing. It is the wrong answer, for three reasons: + +1. **It is invisible to the detector.** The detector reads the *project*. A machine-wide + redirect leaves nothing in the repository to find, so "is this branch safe to merge?" + stops being answerable by any check — which is the exact failure this design exists to + make impossible. +2. **It switches every project at once.** You cannot then have one consumer on a + pre-release and the rest on public releases, which is usually the comparison you want. +3. **A silent fall-back to user-level config is how a pre-release reached a public + registry** during this design's own validation. A tool ignored the config it was handed, + found the user-level file instead, and published for real. Machine-global config is not + a convenience here; it is the loaded gun. + +--- + +## 5. The guards, and why they are load-bearing + +The registry is a public HTTPS endpoint reachable from anywhere. There is no network +boundary doing safety work: **these checks are the containment**, not a second opinion on +top of it. Treat a failure as a build break. + +### In a consumer: `tools/prerelease/detect-prerelease-pins.sh` + +`link` installs it into the consumer at `tools/prerelease/detect-prerelease-pins.sh`. +**Commit it and run it in CI** (and from a pre-commit hook). It flags: + +1. the pre-release registry's host — **defaulted**, so a consumer repo that has never seen + the publisher's config still catches it; +2. any private-network or loopback registry host (someone else's self-hosted instance); +3. a vendor dependency pinned to a pre-release version, in any of the four spellings — the + only signal that survives `pip freeze`, which records no index provenance at all. In + manifests the name and the version share a line, so the match is namespace-anchored + exactly; in **lockfiles** they sit on different lines, so there — and only there — a + proximity window is used instead. Keeping the window out of manifests is deliberate: it + would flag a third-party `rc`/`beta` that merely happens to sit near a vendor entry; +4. an npm dependency declared as a bare dist-tag, which floats; +5. a Maven pom pinning a pre-release in a `` or `` block — the + project's own `1.0.0-SNAPSHOT` version is normal and is deliberately not flagged. + +It scans **dependency declarations only** — manifests and lockfiles. A source file that +binds a test server to `127.0.0.1`, or a design doc quoting an old `-SNAPSHOT`, is not a +dependency on anything, and a check that cries wolf is a check people learn to ignore. + +### In this repo: `scripts/check-no-prerelease-versions.sh` + +Runs in the `gates` lane of `scripts/ci-local.sh` and from `.githooks/pre-commit` whenever a +version-bearing file is staged. A committed pre-release version is not cosmetic: +`scripts/release.mjs` derives the whole lockstep set from the CLI's *current* version, so a +stray `-rc.N` in one `package.json` would silently drop that package from the next real +release. + +### What is deliberately NOT in CI + +There is no pre-release publish workflow in GitHub Actions, and there should not be one. +Publishing needs the write token, and a hosted job holding a token whose only purpose is to +push unreleased artifacts is a standing risk with no matching benefit — the loop it serves +is a developer iterating against a project on their own machine. Pre-release publishing +stays local; the *guards* are what belong in CI. + +--- + +## 6. Relationship to a real release + +`bun run release` (`scripts/release.mjs`) is unchanged and still publishes to the public +registries. `docs/RELEASING.md` keeps its public-npm RC path for the one case a private +registry cannot cover: a release where dependencies or package layout changed, where the +thing being tested *is* a real external install from the real registry. + +Everything else — "does this change work in a downstream app?" — belongs here. + +--- + +## 7. Running your own registry + +The project registry is a normal Gitea instance. To stand up an equivalent (a fork, another +team, an offline machine): + +```bash +docker compose -f tools/prerelease/docker-compose.yml up -d +tools/prerelease/bootstrap.sh # creates the owner + token, writes registry.env +``` + +Then point `MO_REGISTRY_BASE` at it. The publisher and the link helper are registry-agnostic; +nothing else changes. + +> A CDN in front of the registry may cap request bodies (100 MB on Cloudflare's free plan), +> in which case a very large artifact fails at the edge rather than at Gitea. Every artifact +> this repo publishes is far below that. diff --git a/package.json b/package.json index 00f7617c8..6aa234c38 100644 --- a/package.json +++ b/package.json @@ -10,7 +10,8 @@ "typecheck": "bun run --filter '*' typecheck", "test": "cd server/typescript && bun test", "clean": "rm -rf server/typescript/packages/*/dist server/typescript/packages/*/*.tsbuildinfo client/web/packages/*/dist client/web/packages/*/*.tsbuildinfo", - "release": "bun scripts/release.mjs" + "release": "bun scripts/release.mjs", + "prerelease": "bun scripts/prerelease.mjs" }, "devDependencies": { "typescript": "^5.6.0" }, "engines": { "bun": ">=1.3.0", "node": ">=22.0.0" }, diff --git a/scripts/check-no-prerelease-versions.sh b/scripts/check-no-prerelease-versions.sh new file mode 100755 index 000000000..f0e39c962 --- /dev/null +++ b/scripts/check-no-prerelease-versions.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# Guard: a PRE-RELEASE version must never be committed to this repository. +# +# `scripts/prerelease.mjs` edits every version declaration in place and restores them on +# exit, but a crash, a Ctrl-C at the wrong moment, or a hand-run `sed` can leave an +# `-rc.N` / `rc` / `-SNAPSHOT` behind. Committing one is quietly expensive: +# `scripts/release.mjs` derives the whole lockstep set from the CLI's current version +# (`server/typescript/packages/cli/package.json`), so a stray pre-release version silently +# SHRINKS the set that the next real release publishes. +# +# Offline, toolchain-free, runs in milliseconds. Wired into .githooks/pre-commit and the +# `gates` lane of scripts/ci-local.sh. +set -uo pipefail +cd "$(dirname "$0")/.." || exit + +fail=0 +report() { echo " ✖ $1: $2" >&2; fail=1; } + +# npm — every workspace package.json +for f in server/typescript/packages/*/package.json client/web/packages/*/package.json; do + [ -f "$f" ] || continue + v=$(sed -n 's/^[[:space:]]*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$f" | head -1) + case "$v" in *-*) report "$f" "version $v is a pre-release";; esac +done + +# python +v=$(sed -n 's/^version = "\(.*\)"/\1/p' server/python/pyproject.toml | head -1) +case "$v" in *rc*|*dev*|*a[0-9]*|*b[0-9]*) report "server/python/pyproject.toml" "version $v is a pre-release";; esac + +# csharp +v=$(sed -n 's#.*\(.*\).*#\1#p' server/csharp/Directory.Build.props | head -1) +case "$v" in *-*) report "server/csharp/Directory.Build.props" "version $v is a pre-release";; esac + +# java/kotlin — the reactor root decides the line for every module +v=$(grep -m1 -oE '[^<]+' server/java/pom.xml | sed -E 's###g') +case "$v" in *-*) report "server/java/pom.xml" "version $v is a pre-release";; esac + +if [ "$fail" -ne 0 ]; then + cat >&2 <<'MSG' + +Pre-release version(s) found in committed version declarations. +Fix — restore them (scripts/prerelease.mjs does this automatically; a crashed run may not have). +Version declarations only; unrelated WIP under those trees is not touched: + + git checkout -- 'server/typescript/packages/*/package.json' \ + 'client/web/packages/*/package.json' \ + 'server/java/pom.xml' 'server/java/**/pom.xml' \ + 'server/python/pyproject.toml' \ + 'server/csharp/Directory.Build.props' 'bun.lock' +MSG + exit 1 +fi +echo "check-no-prerelease-versions: ✓ no pre-release version in any version declaration" diff --git a/scripts/ci-local.sh b/scripts/ci-local.sh index b573369e1..6b256ace8 100755 --- a/scripts/ci-local.sh +++ b/scripts/ci-local.sh @@ -143,6 +143,13 @@ gate_bun_version() { scripts/check-bun-version.sh; } # them as installable. Offline check; see scripts/check-publish-intent.sh. gate_publish_intent() { scripts/check-publish-intent.sh; } +# ── release hygiene: no pre-release version may be committed ────────────────── +# scripts/prerelease.mjs bumps every version declaration in place and restores them on +# exit; a crashed run (or a hand-run sed) can leave an -rc.N behind. That is not cosmetic: +# scripts/release.mjs derives the lockstep set from the CLI's CURRENT version, so a stray +# pre-release version silently shrinks the set the next real release publishes. +gate_no_prerelease_versions() { scripts/check-no-prerelease-versions.sh; } + # ── peer ranges must have a finite upper bound ──────────────────────────────── # An open `>=` peer silently accepts a future breaking major. `@tanstack/react-table: # ">=8.20.0"` accepted v9 — a rewrite that deleted useReactTable/getCoreRowModel, both @@ -370,6 +377,7 @@ if want gates; then step "leak-scan (security)" gate_leak_scan; if want gates; then step "pom-version parity" gate_pom_versions; fi if want gates; then step "bun-version parity" gate_bun_version; fi if want gates; then step "publish-intent parity" gate_publish_intent; fi +if want gates; then step "no committed pre-release version" gate_no_prerelease_versions; fi if want gates; then step_if bun "peer-range bounds" gate_peer_ranges; fi if want gates; then step_if bun "fixture-lint" gate_fixture_lint; fi # The ts port is split into two lanes so CI can run them as separate jobs (see diff --git a/scripts/prerelease.mjs b/scripts/prerelease.mjs new file mode 100644 index 000000000..6a4f2e567 --- /dev/null +++ b/scripts/prerelease.mjs @@ -0,0 +1,343 @@ +#!/usr/bin/env bun +// Publish a PRE-RELEASE of the in-development version to a PRIVATE registry, so an +// unreleased change can be iterated against downstream consumers without cutting a real +// release on npm / PyPI / NuGet / Maven Central. +// +// bun run prerelease # next iteration, npm (the default scope) +// bun run prerelease --only npm,csharp # pick ports +// bun run prerelease --only all # all four +// bun run prerelease --iter 5 # pin the iteration number +// bun run prerelease --base 0.25.0 # target a different in-development version +// bun run prerelease --dry-run # build + normalize + gate, publish nothing +// +// Registry address and token come from tools/prerelease/registry.env (gitignored) or the +// environment — never from anything committed. See docs/features/prerelease.md. +// +// ── the version scheme ──────────────────────────────────────────────────────────────── +// ONE canonical internal string: -rc. e.g. 0.24.0-rc.3 +// normalized per ecosystem in exactly one place (`V` below): +// +// npm 0.24.0-rc.3 SemVer2, verbatim +// NuGet 0.24.0-rc.3 SemVer2, verbatim +// PyPI 0.24.0rc3 PEP 440 canonical form +// Maven 7.24.0-rc.3 same minor.patch on the historical major 7 +// +// The iteration counter — not a commit sha — is the collision breaker, because npm STRIPS +// SemVer build metadata: `0.24.0-rc.1+aaa` and `0.24.0-rc.1+bbb` compare EQUAL, so the +// second publish is refused as a duplicate. The sha still travels where it is useful: the +// C# packages carry it in AssemblyInformationalVersion via Source Link, which identifies a +// build without participating in resolution. +// +// `-rc.N` was chosen over `-SNAPSHOT` for Maven so all four ports read the same. A mutable +// version is worse than inconvenient: with a lockfile and a warm client cache, re-pushing +// the same version serves the consumer STALE BYTES with no error and no warning. +// +// ── the safety model ────────────────────────────────────────────────────────────────── +// The registry may be on a private network, so "it is only bound to loopback" is NOT the +// guarantee. These are: +// +// 1. The publish target must equal the CONFIGURED registry (or be loopback). Asserted +// before anything runs, as an equality test — not a hostname pattern. +// 2. An independent deny-list of the public registries. Two checks that fail differently +// beat one check trusted twice. +// 3. For npm, `bun publish --dry-run` is PARSED and its reported registry compared to the +// expected one. bun ignores `npm_config_userconfig`; during this design's own +// validation it silently fell back to the user-level ~/.npmrc and published a +// pre-release to the PUBLIC registry. bun is not taken at its word anywhere here. +// 4. HOME is redirected to a scratch dir holding only the private-registry .npmrc, so a +// fall-back has no credential to publish with even if it happens. +// 5. Maven is deployed with an explicit -DaltDeploymentRepository and never -Prelease — +// this repo declares distributionManagement ONLY inside the `release` profile, so a +// bare `mvn deploy` has no target at all. +// 6. Version declarations are edited in place and ALWAYS restored on exit; the script +// refuses to start if any of them is already dirty. +import { execSync, execFileSync } from "node:child_process"; +import { existsSync, mkdirSync, readFileSync, writeFileSync, rmSync } from "node:fs"; +import { join } from "node:path"; + +const argv = process.argv.slice(2); +const flag = (n, d) => { const i = argv.indexOf(`--${n}`); return i === -1 ? d : argv[i + 1]; }; +const has = (n) => argv.includes(`--${n}`); + +const DRY = has("dry-run"); +const ROOT = execSync("git rev-parse --show-toplevel", { encoding: "utf8" }).trim(); +const SCRATCH = join(process.env.TMPDIR || "/tmp", "mo-prerelease"); + +const ok = (m) => console.log(`\x1b[32m✓\x1b[0m ${m}`); +const info = (m) => console.log(` ${m}`); +const die = (m) => { console.error(`\x1b[31m✗ ${m}\x1b[0m`); process.exit(1); }; +const sh = (cmd, o = {}) => execSync(cmd, { encoding: "utf8", cwd: ROOT, stdio: "pipe", ...o }); + +// ── registry config: environment first, then the gitignored env file ────────────────── +const envFile = join(ROOT, "tools/prerelease/registry.env"); +const fileEnv = existsSync(envFile) + ? Object.fromEntries(readFileSync(envFile, "utf8").split("\n") + .filter((l) => l.trim() && !l.trim().startsWith("#") && l.includes("=")) + .map((l) => [l.slice(0, l.indexOf("=")).trim(), l.slice(l.indexOf("=") + 1).trim()])) + : {}; +const cfg = (k) => process.env[k] || fileEnv[k]; + +// The registry HOST is public information and is committed as the default so a consumer +// needs no configuration to resolve pre-releases. The OWNER and the TOKEN are not: the +// token is a credential, and the owner is an account name this public repository does not +// name. Both come from registry.env or the environment. +const DEFAULT_REGISTRY = "https://gitea.mealing.com"; +const BASE = (cfg("MO_REGISTRY_BASE") || DEFAULT_REGISTRY).replace(/\/$/, ""); +const OWNER = cfg("MO_REGISTRY_OWNER"); +const TOKEN = cfg("MO_REGISTRY_TOKEN"); +if (!OWNER || !TOKEN) + die(`publishing needs MO_REGISTRY_OWNER and MO_REGISTRY_TOKEN (reads are anonymous, writes are not) —\n` + + ` set them in the environment, or create ${envFile} from tools/prerelease/registry.env.example`); + +// GATE 1 + 2 — the target must be the configured registry, and must not be a public one. +const PUBLIC_HOSTS = [ + "registry.npmjs.org", "registry.yarnpkg.com", + "pypi.org", "upload.pypi.org", "files.pythonhosted.org", "test.pypi.org", + "api.nuget.org", "www.nuget.org", + "central.sonatype.com", "repo1.maven.org", "repo.maven.apache.org", + "oss.sonatype.org", "s01.oss.sonatype.org", +]; +const expectedHost = new URL(BASE).host; +const assertTarget = (url, what) => { + const u = new URL(url); + if (PUBLIC_HOSTS.includes(u.hostname)) + die(`refusing to publish ${what}: ${u.hostname} is a PUBLIC registry`); + const loopback = ["localhost", "127.0.0.1", "::1"].includes(u.hostname); + if (u.host !== expectedHost && !loopback) + die(`refusing to publish ${what}: ${u.host} is neither the configured registry (${expectedHost}) nor loopback`); + return url; +}; +assertTarget(BASE, "registry base"); + +const NPM_REG = assertTarget(`${BASE}/api/packages/${OWNER}/npm/`, "npm"); +const PYPI_URL = assertTarget(`${BASE}/api/packages/${OWNER}/pypi`, "pypi"); +const NUGET_SRC = assertTarget(`${BASE}/api/packages/${OWNER}/nuget/index.json`, "nuget"); +const MAVEN_URL = assertTarget(`${BASE}/api/packages/${OWNER}/maven`, "maven"); + +// ── version scheme ──────────────────────────────────────────────────────────────────── +const cliPkg = JSON.parse(readFileSync(join(ROOT, "server/typescript/packages/cli/package.json"), "utf8")); +const RELEASED = cliPkg.version; +const BASEVER = flag("base", (() => { + const [maj, min] = RELEASED.split(".").map(Number); + return `${maj}.${min + 1}.0`; +})()); +if (!/^\d+\.\d+\.\d+$/.test(BASEVER)) die(`--base must be a plain x.y.z, got ${BASEVER}`); + +// The lockstep package names, needed before the iteration number is chosen. +const LOCKSTEP = sh("ls -d server/typescript/packages/*/ client/web/packages/*/").trim().split("\n") + .map((d) => d.replace(/\/$/, "")) + .map((dir) => ({ dir, pkg: JSON.parse(readFileSync(join(ROOT, dir, "package.json"), "utf8")) })) + .filter(({ pkg }) => !pkg.private && pkg.version === RELEASED); + +const ITER_RE = new RegExp(`^\\d+\\.${BASEVER.split(".").slice(1).join("\\.")}(?:-rc\\.|rc)(\\d+)$`); + +// Iteration: explicit, else 1 + the highest already taken — across ALL FOUR ecosystems on +// the pre-release registry, so `--only npm` today and `--only csharp` tomorrow cannot +// collide (a used number is a hard 409 on every one of them). +const usedIters = () => { + try { + const list = JSON.parse(sh(`curl -fsS --max-time 10 -u "${OWNER}:${TOKEN}" "${BASE}/api/v1/packages/${OWNER}?limit=1000"`)); + return list.map((p) => (p.version.match(ITER_RE) || [])[1]).filter(Boolean).map(Number); + } catch { return []; } +}; + +// ...and across PUBLIC npm as well. A version published there is permanent — unpublish is +// refused outright once anything depends on it, and deprecation does not free the number. +// `@metaobjectsdev/metadata@0.24.0-rc.1` is burned exactly that way. Reusing a burned +// number privately costs nothing today and guarantees a failure the day that iteration is +// promoted to a public RC, so the counter skips it here instead of failing there. +const burnedPublicIters = async () => { + const results = await Promise.all(LOCKSTEP.map(async ({ pkg }) => { + try { + const r = await fetch(`https://registry.npmjs.org/${pkg.name.replace("/", "%2f")}`, + { headers: { accept: "application/vnd.npm.install-v1+json" }, signal: AbortSignal.timeout(10_000) }); + if (!r.ok) return []; + const doc = await r.json(); + return Object.keys(doc.versions || {}).map((v) => (v.match(ITER_RE) || [])[1]).filter(Boolean).map(Number); + } catch { return null; } // offline: fail OPEN, the release preflight is the backstop + })); + if (results.some((r) => r === null)) + console.warn(`\x1b[33m! could not reach public npm — the iteration number may reuse one that is already burned there\x1b[0m`); + return results.filter(Boolean).flat(); +}; + +const explicitIter = flag("iter", null); +const burned = await burnedPublicIters(); +const ITER = Number(explicitIter ?? (() => { + const taken = [...usedIters(), ...burned]; + return taken.length ? Math.max(...taken) + 1 : 1; +})()); +if (!Number.isInteger(ITER) || ITER < 1) die(`--iter must be a positive integer, got ${ITER}`); +if (explicitIter && burned.includes(ITER)) + console.warn(`\x1b[33m! ${BASEVER}-rc.${ITER} is already published on PUBLIC npm for at least one package.\n` + + ` Publishing it privately is fine, but that number can never be used for a public RC.\x1b[0m`); + +const CANON = `${BASEVER}-rc.${ITER}`; +const MAVEN_MAJOR = 7; +// The ONE place per-ecosystem normalization lives. +const V = { + npm: CANON, + nuget: CANON, + pypi: `${BASEVER}rc${ITER}`, + maven: `${MAVEN_MAJOR}.${BASEVER.split(".").slice(1).join(".")}-rc.${ITER}`, +}; + +const ALL = ["npm", "python", "csharp", "java"]; +const onlyArg = flag("only", "npm"); +const ONLY = onlyArg === "all" ? ALL : onlyArg.split(",").map((s) => s.trim()); +const unknown = ONLY.filter((p) => !ALL.includes(p)); +if (unknown.length) die(`--only expects ${ALL.join("|")}|all, got '${unknown.join(",")}'`); +const wants = (p) => ONLY.includes(p); + +console.log(`\n── pre-release ${CANON}${DRY ? " (DRY RUN)" : ""} → ${expectedHost} ──\n`); +info(`released: ${RELEASED} base: ${BASEVER} iteration: ${ITER}`); +info(`npm ${V.npm} · pypi ${V.pypi} · nuget ${V.nuget} · maven ${V.maven}`); +info(`ports: ${ONLY.join(", ")}\n`); +ok(`publish target verified: ${expectedHost} (configured registry; not a public one)`); + +// ── GATE 6 — version-bearing files must be clean, so restore is unambiguous ──────────── +const VERSION_FILES = [ + "server/typescript/packages/*/package.json", "client/web/packages/*/package.json", + "server/java/pom.xml", "server/java/**/pom.xml", + "server/python/pyproject.toml", "server/csharp/Directory.Build.props", + "bun.lock", +]; +const dirty = sh(`git status --porcelain -- ${VERSION_FILES.map((f) => `'${f}'`).join(" ")}`).trim(); +if (dirty) die(`version-bearing files are dirty — commit or stash first:\n${dirty}`); +ok("version-bearing files clean"); +// The tracked files those same pathspecs cover — restore() may only revert files whose +// pre-run cleanliness this gate actually verified. +const covered = new Set( + sh(`git ls-files -- ${VERSION_FILES.map((f) => `'${f}'`).join(" ")}`).trim().split("\n").filter(Boolean)); + +// Only the files this run actually wrote are reverted — never whole trees, or a maintainer's +// unrelated unstaged WIP anywhere under them would be silently destroyed on exit (even on a +// clean --dry-run). GATE 6 guarantees each of these was clean before the run, so restoring +// from the index is unambiguous. +let restored = false; +const touched = []; +const touch = (...files) => { + for (const f of files) + if (!covered.has(f)) + die(`about to mutate ${f}, which the clean-tree gate never covered — restoring it could destroy uncommitted work`); + touched.push(...files); +}; +const restore = () => { + if (restored) return; restored = true; + if (!touched.length) return; + try { sh(`git checkout -- ${touched.map((f) => `'${f}'`).join(" ")}`); } catch {} +}; +process.on("exit", restore); +process.on("SIGINT", () => { restore(); process.exit(130); }); + +const scratch = (name) => { const d = join(SCRATCH, name); rmSync(d, { recursive: true, force: true }); mkdirSync(d, { recursive: true }); return d; }; +// Kept between runs (so Maven does not re-download the world every time) and deliberately +// NOT the user's ~/.m2: a pre-release must never land in the local repository that ordinary +// builds resolve from, or an unrelated `mvn` picks it up with no way to tell. +const keptScratch = (name) => { const d = join(SCRATCH, name); mkdirSync(d, { recursive: true }); return d; }; + +// ── npm ─────────────────────────────────────────────────────────────────────────────── +if (wants("npm")) { + const set = []; + for (const { dir, pkg } of LOCKSTEP) { // same lockstep rule as scripts/release.mjs + const p = { ...pkg, version: V.npm }; + const f = join(dir, "package.json"); + touch(f); + writeFileSync(join(ROOT, f), JSON.stringify(p, null, 2) + "\n"); + set.push({ dir, short: p.name.replace("@metaobjectsdev/", "") }); + } + if (!set.length) die("no packages matched the lockstep set"); + ok(`npm lockstep set: ${set.length} packages → ${V.npm}`); + + touch("bun.lock"); + sh("rm -f bun.lock && bun install"); // re-pins workspace:* to V.npm + sh("bun run clean && bun run build"); + ok("relocked + clean rebuild"); + + const p = scratch("pack"); + sh(`cd server/typescript/packages/cli && bun pm pack --destination ${p}`); + const pj = JSON.parse(sh(`tar -xzOf ${p}/*.tgz package/package.json`)); + const bad = Object.entries(pj.dependencies || {}).filter(([k, v]) => k.startsWith("@metaobjectsdev/") && v !== V.npm); + if (bad.length) die(`packed cli pins stale sibling deps: ${JSON.stringify(bad)}`); + ok(`packed deps pinned to ${V.npm}`); + + const home = scratch("npmhome"); + writeFileSync(join(home, ".npmrc"), + `@metaobjectsdev:registry=${NPM_REG}\n${NPM_REG.replace(/^https?:/, "")}:_authToken=${TOKEN}\n`); + + const TIERS = ["metadata", "render", "codegen-ts", "runtime-ts", "migrate-ts", "sdk", "docs-site", + "runtime-web", "codegen-ts-react", "codegen-ts-tanstack", "react", "tanstack", "cli", "ai-runtime"]; + const ordered = [...set].sort((a, b) => TIERS.indexOf(a.short) - TIERS.indexOf(b.short)); + + // GATE 3 — ask bun where it would actually publish, and believe only that. + const probe = execFileSync("bun", ["publish", "--dry-run"], + { cwd: join(ROOT, ordered[0].dir), env: { ...process.env, HOME: home }, encoding: "utf8" }); + const seen = (probe.match(/^Registry:\s*(\S+)/m) || [])[1]; + if (seen !== NPM_REG) die(`bun would publish to ${seen}, not ${NPM_REG} — aborting`); + ok(`bun publish target verified by dry-run: ${seen}`); + + if (!DRY) for (const pkg of ordered) { + execFileSync("bun", ["publish", "--tag", "prerelease"], + { cwd: join(ROOT, pkg.dir), env: { ...process.env, HOME: home }, stdio: "pipe" }); + info(`published ${pkg.short}@${V.npm}`); + } + ok(DRY ? "npm: dry run, nothing published" : `npm: ${ordered.length} packages @ ${V.npm} (dist-tag prerelease)`); +} + +// ── python ──────────────────────────────────────────────────────────────────────────── +if (wants("python")) { + const f = "server/python/pyproject.toml"; + touch(f); + writeFileSync(join(ROOT, f), readFileSync(join(ROOT, f), "utf8").replace(/^version = ".*"$/m, `version = "${V.pypi}"`)); + const dist = scratch("pydist"); + sh(`cd server/python && uv build --out-dir ${dist}`); + ok(`python built ${V.pypi}`); + if (!DRY) { + sh(`cd server/python && uv publish --publish-url "${PYPI_URL}" --username "${OWNER}" --password "${TOKEN}" ${dist}/*`); + ok(`python: metaobjects ${V.pypi} → ${expectedHost}`); + } else ok("python: dry run, nothing published"); +} + +// ── csharp ──────────────────────────────────────────────────────────────────────────── +if (wants("csharp")) { + // No file edit needed — dotnet takes the version on the command line. + const out = scratch("nupkg"); + const projs = ["MetaObjects/MetaObjects", "MetaObjects.Render/MetaObjects.Render", + "MetaObjects.Codegen/MetaObjects.Codegen", "MetaObjects.Cli/MetaObjects.Cli"]; + for (const p of projs) sh(`cd server/csharp && dotnet pack ${p}.csproj -c Release -o ${out} -p:Version=${V.nuget} --nologo -v q`); + ok(`csharp packed ${V.nuget} (4 packages)`); + if (!DRY) { + sh(`dotnet nuget push "${out}/*.nupkg" --source "${NUGET_SRC}" --api-key "${TOKEN}"`); + ok(`csharp: 4 packages @ ${V.nuget} → ${expectedHost}`); + } else ok("csharp: dry run, nothing published"); +} + +// ── java / kotlin ───────────────────────────────────────────────────────────────────── +if (wants("java")) { + const javaReleased = readFileSync(join(ROOT, "server/java/pom.xml"), "utf8").match(/([^<]+)<\/version>/)[1]; + // Tree-wide sed, NOT `mvn versions:set` — versions:set walks only the reactor and + // silently skips the two reactor-EXCLUDED integration-test modules, whose parent + // version then lags (see scripts/check-pom-versions.sh, docs/RELEASING.md). + const poms = sh(`grep -rl '${javaReleased}' --include=pom.xml server/java`).trim().split("\n").filter(Boolean); + touch(...poms); + sh(`sed -i 's/${javaReleased.replace(/\./g, "\\.")}/${V.maven}/g' ${poms.map((f) => `'${f}'`).join(" ")}`); + sh("scripts/check-pom-versions.sh"); + ok(`java poms → ${V.maven}`); + const settings = join(scratch("m2"), "settings.xml"); + writeFileSync(settings, + `` + + `mo-prerelease${OWNER}${TOKEN}` + + `\n`); + // `package` for a dry run, not `install`: a dry run must not write artifacts anywhere. + const goal = DRY ? "package" : "deploy"; + const repo = keptScratch("m2repo"); + sh(`cd server/java && mvn -B -s ${settings} -Dmaven.repo.local=${repo} ${goal} -DskipTests` + + ` -DaltDeploymentRepository="mo-prerelease::${MAVEN_URL}"`, + { stdio: "inherit" }); + ok(DRY ? "java: dry run (install only)" : `java: reactor @ ${V.maven} → ${expectedHost}`); +} + +restore(); +console.log(`\n\x1b[32m\x1b[1m✅ ${CANON} is on the pre-release registry.\x1b[0m`); +console.log(` link a consumer: tools/prerelease/prerelease-link.sh link --project --version ${CANON}`); +console.log(` consumer guide: docs/features/prerelease.md\n`); diff --git a/scripts/release.mjs b/scripts/release.mjs index ba7ebfb9f..7c5524349 100644 --- a/scripts/release.mjs +++ b/scripts/release.mjs @@ -67,15 +67,33 @@ const dirty = out("git status --porcelain").split("\n") .filter((l) => l && !l.startsWith("??") && !l.endsWith("CHANGELOG.md")); if (dirty.length) die(`uncommitted changes:\n${dirty.join("\n")}\n(commit or stash first; CHANGELOG is allowed)`); -// Target version free on npm + no existing tag. -try { sh(`npm view @metaobjectsdev/cli@${VERSION} version`, { quiet: true }); die(`${VERSION} is already published`); } -catch { /* 404 = free, good */ } if (out(`git tag -l v${VERSION}`)) die(`tag v${VERSION} already exists`); // The lockstep set = every non-private package at the CURRENT version (cli's version). const current = pkgs.find((p) => p.short === "cli").pkg.version; const set = pkgs.filter((p) => !p.pkg.private && p.pkg.version === current); -ok(`preflight: on main, synced, ${VERSION} free`); + +// The target version must be free for EVERY package in the set, not just the cli. +// Checking one package is a late failure waiting to happen: npm versions are permanent +// (unpublish is refused outright once anything depends on the version, and deprecating it +// does not free the number), so a version burned on a single package — as +// @metaobjectsdev/metadata@0.24.0-rc.1 is — would fail mid-publish, after its dependencies +// had already shipped irreversibly. Checked in parallel: 14 sequential `npm view`s is 14s. +const taken = (await Promise.all(set.map(async (p) => { + try { + const r = await fetch(`https://registry.npmjs.org/${p.pkg.name.replace("/", "%2f")}`, + { headers: { accept: "application/vnd.npm.install-v1+json" }, signal: AbortSignal.timeout(15_000) }); + if (r.status === 404) return null; // never published = free + // Any other non-OK status is fatal, not "free": failing open here re-creates the + // irreversible mid-publish partial failure this preflight exists to prevent. + if (!r.ok) die(`npm answered HTTP ${r.status} for ${p.pkg.name} — cannot verify ${VERSION} is free`); + return (await r.json()).versions?.[VERSION] ? p.pkg.name : null; + } catch { die(`could not reach npm to verify ${VERSION} is free (${p.pkg.name})`); } +}))).filter(Boolean); +if (taken.length) + die(`${VERSION} is already published for:\n ${taken.join("\n ")}\n` + + `npm versions are permanent — pick the next free version.`); +ok(`preflight: on main, synced, ${VERSION} free on all ${set.length} packages`); ok(`lockstep set @ ${current}: ${set.length} packages → ${VERSION}`); // --- PHASE 1: bump -------------------------------------------------------- diff --git a/tools/prerelease/bootstrap.sh b/tools/prerelease/bootstrap.sh new file mode 100755 index 000000000..8b9f24606 --- /dev/null +++ b/tools/prerelease/bootstrap.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# One-time setup for the OPTIONAL self-contained registry in docker-compose.yml. +# +# Creates the registry owner account, mints a package-scoped token, and writes +# tools/prerelease/registry.env (gitignored). Skip this entirely if you already have a +# registry — just copy registry.env.example to registry.env and fill it in. +# +# docker compose -f tools/prerelease/docker-compose.yml up -d +# tools/prerelease/bootstrap.sh +# +# Re-running is safe: it reuses the account and mints a fresh token. +set -euo pipefail + +HOST="${MO_REGISTRY_BASE:-http://localhost:3939}" +OWNER="${MO_REGISTRY_OWNER:-prerelease}" +CONTAINER="${MO_REGISTRY_CONTAINER:-mo-prerelease-registry}" +HERE="$(cd "$(dirname "$0")" && pwd)" +ENV_FILE="$HERE/registry.env" + +# This script drives the compose container directly, so it only makes sense against the +# instance that compose file starts. +case "$HOST" in + http://localhost:*|http://127.0.0.1:*) ;; + *) echo "bootstrap.sh only manages the local compose instance; for an existing registry" >&2 + echo "copy registry.env.example to registry.env and fill in your own values." >&2 + exit 1 ;; +esac + +# No committed default for an --admin credential — this repository is public, so a +# publicly-known password would exist behind nothing but the loopback bind. MO_REGISTRY_PASS +# wins; else a previous bootstrap's registry.env is reused (same registry only) so +# re-running stays idempotent; else a fresh random password is minted and persisted below. +PASS="${MO_REGISTRY_PASS:-}" +if [ -z "$PASS" ] && [ -f "$ENV_FILE" ]; then + prev_base="$(sed -n 's/^MO_REGISTRY_BASE=//p' "$ENV_FILE" | tail -n1)" + if [ "$prev_base" = "$HOST" ]; then + PASS="$(sed -n 's/^MO_REGISTRY_PASS=//p' "$ENV_FILE" | tail -n1)" + fi +fi +if [ -z "$PASS" ]; then + PASS="$(openssl rand -base64 24 2>/dev/null || head -c 24 /dev/urandom | od -An -tx1 | tr -d ' \n')" + [ -n "$PASS" ] || { echo "could not generate a password — set MO_REGISTRY_PASS" >&2; exit 1; } +fi + +echo "waiting for $HOST ..." +for _ in $(seq 1 60); do + if curl -fsS -o /dev/null "$HOST/"; then + break + fi + sleep 1 +done + +if ! curl -fsS -o /dev/null -u "$OWNER:$PASS" "$HOST/api/v1/user" 2>/dev/null; then + docker exec -u git "$CONTAINER" gitea admin user create \ + --username "$OWNER" --password "$PASS" --email "$OWNER@registry.invalid" \ + --admin --must-change-password=false >/dev/null + echo "created registry owner '$OWNER'" +fi + +TOKEN="$(curl -fsS -X POST -H 'Content-Type: application/json' -u "$OWNER:$PASS" \ + -d '{"name":"prerelease-'"$(date +%s)"'","scopes":["write:package","read:package"]}' \ + "$HOST/api/v1/users/$OWNER/tokens" | sed -n 's/.*"sha1":"\([^"]*\)".*/\1/p')" + +[ -n "$TOKEN" ] || { echo "failed to mint a token" >&2; exit 1; } + +cat > "$ENV_FILE" < or block. The +# groupId and the version live on different lines (or behind a property), so a +# same-line match cannot see it. The project's OWN 1.0.0-SNAPSHOT version is normal +# and is deliberately not flagged. +# +# Pure text scan: no network, no package manager, no toolchain. +set -uo pipefail + +ROOT="${1:-.}" + +# The vendor namespaces whose pre-release pins matter. Edit for your own scopes. +NS_RE='@metaobjectsdev/|com\.metaobjects|(^|[^A-Za-z])MetaObjects(\.|"|<|$)|(^|[^A-Za-z-])metaobjects([^A-Za-z-]|$)' + +# The pre-release registry's host, known by default so this check needs no configuration +# in a consumer repo — which matters, because the consumer is exactly where nobody has the +# publisher's config. MO_REGISTRY_BASE overrides it for a different registry. +DEFAULT_REGISTRY_HOST='gitea.mealing.com' +CFG="$(cd "$(dirname "$0")" && pwd)/registry.env" +# shellcheck source=/dev/null +[ -f "$CFG" ] && . "$CFG" +REGISTRY_HOST="$DEFAULT_REGISTRY_HOST" +if [ -n "${MO_REGISTRY_BASE:-}" ]; then + h="${MO_REGISTRY_BASE#*://}"; h="${h%%/*}" + [ "$h" = "$DEFAULT_REGISTRY_HOST" ] || REGISTRY_HOST="$DEFAULT_REGISTRY_HOST|$h" +fi + +# Private/loopback/link-local hosts, and the suffixes used for LAN-only names. +PRIVATE_HOST_RE='https?://(localhost|127\.[0-9]+\.[0-9]+\.[0-9]+|0\.0\.0\.0|10\.[0-9]+\.[0-9]+\.[0-9]+|192\.168\.[0-9]+\.[0-9]+|172\.(1[6-9]|2[0-9]|3[01])\.[0-9]+\.[0-9]+|169\.254\.[0-9]+\.[0-9]+|\[::1\]|[A-Za-z0-9._-]+\.(local|lan|internal|home|localdomain))(:[0-9]+)?' + +# A pre-release version in any of the four spellings we emit. +# npm / NuGet / Maven 0.24.0-rc.3 · 7.24.0-rc.3 · x-SNAPSHOT · -alpha/-beta +# PEP 440 0.24.0rc3 · 0.24.0.dev1 +PRERELEASE_RE='[0-9]+\.[0-9]+\.[0-9]+(-(rc|alpha|beta|SNAPSHOT)[.0-9]*|rc[0-9]+|\.dev[0-9]+)' +# awk builds its regex from a string, so every backslash has to survive one more round. +PRERELEASE_RE_AWK='[0-9]+\\.[0-9]+\\.[0-9]+(-(rc|alpha|beta|SNAPSHOT)[.0-9]*|rc[0-9]+|\\.dev[0-9]+)' +# NS_RE with every backslash doubled for the same reason (see the proximity check below). +NS_RE_AWK='@metaobjectsdev/|com\\.metaobjects|(^|[^A-Za-z])MetaObjects(\\.|\"|<|$)|(^|[^A-Za-z-])metaobjects([^A-Za-z-]|$)' + +# Only DEPENDENCY DECLARATIONS are scanned. A source file that starts an HTTP server on +# 127.0.0.1, or a design doc quoting an old -SNAPSHOT version, is not a dependency on +# anything — scanning those produced nothing but noise, and a check that cries wolf is a +# check people learn to ignore. +MANIFESTS=( + # npm + --include=package.json --include=package-lock.json --include=npm-shrinkwrap.json + --include=yarn.lock --include=pnpm-lock.yaml --include=bun.lock --include=.npmrc + --include=.yarnrc.yml + # python + --include=pyproject.toml --include='requirements*.txt' --include='constraints*.txt' + --include=uv.lock --include=poetry.lock --include=Pipfile --include=Pipfile.lock + --include=setup.cfg --include=pip.conf --include=.pypirc + # nuget + --include='*.csproj' --include='*.fsproj' --include='*.vbproj' + --include='Directory.*.props' --include='Directory.*.targets' + --include=NuGet.config --include=nuget.config --include=packages.lock.json + --include=packages.config --include=paket.dependencies + # maven / gradle + --include=pom.xml --include='build.gradle' --include='build.gradle.kts' + --include=gradle.properties --include=settings.xml --include=libs.versions.toml +) + +# Arm 3b below applies ONLY to these. In a lockfile the package name and its version sit +# on different lines, so a same-line namespace match cannot see the pair and a proximity +# window is the only way to read it. Every other manifest format keeps name and version on +# ONE line, where arm 3a already reads them exactly — running the window there instead +# flags a third-party beta that merely happens to sit near a vendor entry, which is the +# cry-wolf failure this check must not have. +LOCKFILES=( + --include=package-lock.json --include=npm-shrinkwrap.json --include=yarn.lock + --include=pnpm-lock.yaml --include=bun.lock + --include=uv.lock --include=poetry.lock --include=Pipfile.lock + --include=packages.lock.json +) + +EXCLUDES=( + --exclude-dir=.git --exclude-dir=node_modules --exclude-dir=vendor + --exclude-dir=".venv*" --exclude-dir=venv --exclude-dir=site-packages + --exclude-dir=.tox --exclude-dir=.mypy_cache --exclude-dir=__pycache__ + --exclude-dir=target --exclude-dir=bin --exclude-dir=obj --exclude-dir=dist + --exclude-dir=build --exclude-dir=.gradle --exclude-dir=.next --exclude-dir=.nuxt +) + +fail=0 +hit() { echo " ✖ $1" >&2; fail=1; } + +scan() { # scan