feat(sdk): add metadata source resolution and scope filtering - #311
Merged
Conversation
Sources become a SET rather than an ordered list: super-resolution is already order-independent (#188) and the loader already discards the caller's order in _partitionOverlayLast, so declared order carries no information the loader consumes. That deletes topological sorting, cycle detection, and the diamond-dependency problem from the design. Scope is package patterns applied at OUTPUT, never to input — a partial input file list can fail to load when an extends target is missing, so input-side subsetting is wrong by construction. Prior art is open-source only, every claim carrying a public-docs URL, with licenses noted per project and hosted/commercial components excluded. Phase 1 is config-only and additive; a project with one root config and no sources declared keeps byte-identical output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… all ports Register ERR_SOURCE_UNRESOLVED, ERR_SOURCE_KIND_UNSUPPORTED, ERR_SCOPE_PATTERN_INVALID, and ERR_COLLECTION_NOT_FOUND across all five language ports (TS, Python, Java, C#) and the fixtures/conformance/ERROR-CODES.json ledger. These codes will be raised by later tasks when loading metadata sources from .metaobjects/config.json. Descriptions corrected to match the design: - ERR_SOURCE_UNRESOLVED: a path source on disk does not exist - ERR_SOURCE_KIND_UNSUPPORTED: source kind (resource/package) not supported by toolchain - ERR_SCOPE_PATTERN_INVALID: scope include/exclude package pattern is malformed - ERR_COLLECTION_NOT_FOUND: no metadata collection discovered (no sources config, no default metaobjects/ dir) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… more) Phase-1 metadata-source-resolution, task 1. A pure, no-I/O module deciding whether a fully-qualified node name falls inside a consumer's declared include/exclude scope, ahead of the source-resolution and discovery work that builds on it. Raises ERR_SCOPE_PATTERN_INVALID via ParseError + codeSource (this repo's loader-error convention), not a bare Error with the code embedded in the message text. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…-port Pins the scope-engine pattern semantics (compileScope/matchesScope, task 1) as a shared cross-port corpus so `*`/`**` cannot drift into five different meanings per language port -- the same failure class as the LIKE/ILIKE divergence fixed in 0.21.6. Adds a mid-pattern `**` case (acme::**::Order) the task-1 review flagged as untested; verified against the real implementation that a zero-segment gap is correctly rejected (acme::Order does not match), consistent with the "one or more segments" rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
resolveSources turns a declared source SET into a sorted, de-duplicated list of metadata file paths — sorted by absolute path so permuting the input specs cannot change the result. Only `path` specs resolve in phase 1 (`resource`/`package` throw ERR_SOURCE_KIND_UNSUPPORTED); an unresolvable path throws ERR_SOURCE_UNRESOLVED rather than silently contributing nothing. File collection uses stat (not lstat or Dirent.isDirectory()) so a symlinked subdirectory is traversed, matching DirectorySource in @metaobjectsdev/metadata. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two overlapping specs contributing the same file previously kept "first spec processed wins," so `.file` was permutation-safe but the attributed `.spec` on that entry was not — a later phase-1 task's order-independence gate deep-equals the full ResolvedSource[] and would have failed on any overlapping-source project. The tie-break is now content-only (smaller JSON.stringify(spec) wins), so the full result is a pure function of the source set regardless of processing order. Also makes the metadata-file extension match case-insensitive (extname().toLowerCase()), matching DirectorySource in @metaobjectsdev/metadata exactly — a meta.JSON file was previously picked up by the loader and silently skipped here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Widens ConfigSchema.sources from the dead 2-arm kind-discriminated union to
the 3-arm path/resource/package shape from SourceSpec (sources.ts), adds an
optional top-level scope block mirroring Scope (scope.ts), and adds
migrate.scope for restricting a migrate run to a subset of loaded metadata.
SourceSpecSchema deliberately skips .strict() on its arms: an existing
project's config.json may still carry the pre-phase-1 { kind: "path", path:
"..." } shape, and .strict() would reject the extra kind key and break that
config on the next run. Zod's default strip-unknown-keys behavior parses it
under the modern { path: "..." } shape instead, keeping the pre-existing
kind-shaped tests in config.test.ts green untouched.
Adds a compile-time parity assertion (a direct assignment, not a conditional
type — the latter silently resolves to never on drift instead of erroring)
so SourceSpecSchema's inferred type and the hand-written SourceSpec can't
drift unnoticed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…typos
Ruling reversal on the prior commit: the pre-phase-1 { kind: "path", path:
"..." } shape never shipped to an adopter (meta init has only ever
scaffolded "sources": [], and nothing under src/ ever read it) — it existed
only in this package's own tests. There is no live config to be lenient
for, so the back-compat concern that motivated dropping .strict() doesn't
hold, and this project is fail-closed on undeclared keys everywhere else
(ADR-0023). Restores .strict() on all three SourceSpecSchema arms and
updates the four legacy { kind, ... } fixture shapes (sdk/test/config.test.ts,
cli/test/init.test.ts) to the modern shape, same assertion intent. Adds a
test pinning the strictness itself: an unrecognized sibling key on a source
is now a hard parse error rather than being silently stripped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Walk up from a starting directory for the nearest .metaobjects/config.json, stopping after examining a directory containing .git so a monorepo can never silently adopt a parent checkout's configuration. The config check runs before the .git check within each directory, so a repo-root config (sharing its directory with .git) stays reachable from any subdirectory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Composes discovery, config, source resolution and scope into the single function every metadata read path routes through. `metaobjects/` is the DEFAULT value of `sources`, never a requirement — no call site downstream of this may assume the directory name. Uses ParseError + a `code` property (this codebase's error convention), not a message-prefixed plain Error, and wires scope/sources/discovery into the package barrel alongside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lows it A config.json that EXISTS but fails to load (malformed JSON, a ConfigSchema violation) was being caught and silently treated as "no config declared", falling through to DEFAULT_SOURCES. That let a typo'd config quietly generate from a possibly-stale metaobjects/ with no diagnostic — a worse failure than the one this design exists to remove, since resolveCollection is the single authority every command routes through. Fixed by checking for the config FILE (fileExists), not the .metaobjects directory (isDir told us nothing about whether config.json was actually inside it) — an absent file still falls through silently (the ordinary "no config" case), but a present-and-broken one now propagates uncaught. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…te-identically
The linchpin gate for the source-resolution design: resolution is a pure
function of the declared source SET, so declaration order carries no
information. Loads a base + an overlay onto it + an independent third file
across all six permutations of the three specs, and asserts:
1. resolveSources() output is deep-equal (full ResolvedSource[], .spec
included, not just .file) across every permutation.
2. The loaded model's own-mode canonicalSerialize() output is
byte-identical across every permutation.
Also pins that the permutations() helper genuinely produces 6 distinct
orderings, not 6 copies of the same one.
Verified the gate is real by temporarily removing resolveSources' .sort()
(sources.ts unmodified in this commit) — both assertions failed with a
diff naming the exact diverging permutation, confirming the test actually
depends on order-independent behavior rather than passing vacuously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…bling order Second correction to the order-independence gate (task 8). Review found the first fix overcorrected: comparing whole-root canonicalSerialize() output across permuted FileSource[] input failed on unmodified code, but only because MetaRoot's top-level children array follows raw parse order — Order vs Customer swapping position depending on which file the loader saw first. That was never a design claim: canonicalSerialize()'s own contract promises alphabetical @-attr keys and a trailing newline, nothing about sibling ordering, and production never hands the loader a permuted list — resolveSources() sorts by absolute path first (test 1). Test 2 now compares CONTENT, not the whole tree: a name-keyed Map<string, string> of each top-level object's own canonicalSerialize(), built per permutation and deep-equal'd across all six — insensitive to sibling order by construction, but still catches any real content divergence. It also directly asserts the overlay's `note` field landed on Order in every permutation, since that's the specific fact _partitionOverlayLast is responsible for. File header rewritten to document the three-layer truth this gate now encodes: resolveSources canonicalizes file order (test 1); the loader resolves content order-independently including overlay-before-base (test 2); sibling order of unrelated top-level nodes follows input order and is deliberately not asserted, because production never permutes. Break-and-revert re-confirmed against the rewritten test: commenting out _partitionOverlayLast's call in meta-data-loader.ts makes permutation [b, a, c] (overlay before base) throw ERR_OVERLAY_NO_TARGET, caught by the per-permutation result.errors assertion; reverted immediately (meta-data-loader.ts is byte-identical to HEAD in this commit — the diff touches only the test file). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ns, error-code order, strict config, parity guard Five verified defects in the phase-1 metadata-source-resolution module set: - sources.ts collectDir: a dangling symlink (or TOCTOU removal/EACCES) inside a walked directory crashed resolveSources with a raw Node ENOENT carrying no ERR_ code. Now caught and skipped, matching DirectorySource in @metaobjectsdev/metadata, which this walk already claims to mirror. A declared path spec that itself doesn't resolve still throws ERR_SOURCE_UNRESOLVED — unchanged, tested. - scope.ts compilePattern: an odd colon run (e.g. "acme:::Order") survived the split on the two-char "::" separator with a leftover ":" inside a segment, compiling to a regex no legal fully-qualified name could ever match — a typo'd include pattern silently scoped out everything. Now rejected as ERR_SCOPE_PATTERN_INVALID. - sources.ts resolveSources: kind validation and path resolution were interleaved in one loop, so an unsupported source kind was reported only if no earlier spec's path failed to resolve first — the error code depended on declaration order, contradicting the module's own pure-function-of-the-SET invariant. Kind is now validated for every spec up front, before any filesystem I/O. - config.ts ConfigSchema: strict() on the source-spec arms and the scope block but not the enclosing object, so a misspelled top-level key (e.g. "scopes") was silently stripped and the collection resolved as "everything in scope" — the exact fail-open .strict() elsewhere exists to prevent, one level up. Now strict() at the top level too; audited every call site (cli/src/commands/init.ts, cli/src/lib/config.ts) — none passes an extra key, and the cli suite stays at the same 556/3/0 baseline. - config.ts _sourceSpecParity: a one-directional assignment only proved the Zod schema's inferred type assignable to the hand-written SourceSpec, so an arm added to SourceSpec without a matching schema arm still compiled clean. Now bidirectional (two assignments, opposite directions); each half was deliberately broken and reverted to confirm it fires (see the quality-pass report for both red tsc outputs). Also carries two changes that are compile-coupled to this file set via a new sources.ts -> memory.ts import edge (DEFAULT_SOURCES no longer hardcodes "metaobjects" as a second encoding of memory.ts's DEFAULT_METADATA_DIR; the package's two metadata-file walkers now share one case-insensitive isMetadataFile owned by memory.ts, aligning sources.ts's already-fixed DirectorySource-matching behavior into memory.ts's loadMemory walk too — a real behavior change, pinned by a new test) — full rationale, including a concrete crash repro for why the ownership direction differs from the originally-requested one, is in the quality-pass report (not committed; listed in .git/info/exclude). Verified: bun test packages/sdk 219/0 (was 214/0), bun test packages/cli 556/3/0 (unchanged), sdk + cli typecheck clean, all confirmed in isolation (stashed the remaining reuse/efficiency changes before running these). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… boolean
Reuse and one small efficiency fix flagged by the quality pass, confined to
discovery.ts and collection.ts:
- collection.ts's fileExists was byte-identical to discovery.ts's exists.
discovery.ts's exists is now exported and imported instead of duplicated;
collection.ts's isDir stays local (a genuinely different predicate).
- CONFIG_FILE ("config.json") was declared three times: config.ts (private),
discovery.ts, and collection.ts (with a comment justifying the
duplication because config.ts didn't export it). config.ts now exports it
as the single owner; discovery.ts and collection.ts import it, and
collection.ts's justifying comment is gone along with the duplicate.
- collection.ts re-stat'd .metaobjects/config.json on the discovered-dir
path even though findConfigDir (discovery.ts) had already proved its
presence/absence to return its result. hasConfig is now derived directly
from findConfigDir's return value on that path; the stat only still runs
on the explicitDir path, where findConfigDir never executes and nothing
else has proven the file's existence.
Deliberately left alone: collection.ts's isDir pre-flight before the
ERR_COLLECTION_NOT_FOUND throw is a second stat that exists purely to
produce a clearer diagnostic than the raw ERR_SOURCE_UNRESOLVED
resolveSources would otherwise throw — trading that for one syscall is a
bad trade, so it's untouched (now with a comment saying so explicitly).
Verified: bun test packages/sdk 219/0, sdk typecheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add `files?: readonly string[]` to LoadMemoryOptions. When supplied, loadMemory loads exactly those files and skips all directory discovery (no metaobjects/ scan, no workspace extends: walk). Absent, behavior is unchanged — collectMetadataPaths/listMetadataFiles stay the fallback and back-compat path. Later tasks route the CLI's read sites through resolveCollection() into this option. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eCollection
`sources` in .metaobjects/config.json becomes the real authority on where
metadata lives; `metaobjects/` becomes merely its default. Five hardcoded
reads now route through `resolveCollection()` (from Task 7) and pass its
resolved file list to `loadMemory({ files })` (Task 9):
- gen.ts — the existsSync(metaobjects/) hint branch is deleted; discovery and
load stay two separate try blocks so a genuine ParseError is never
misreported as "no metaobjects/ found".
- docs.ts — the markdown-surface load, and the standalone HTML site's own
loader (which wants directories, not a file list; new collectionSourceDirs
helper derives them from the collection's distinct source specs).
- export.ts — resolveCollection failures fold into the same exit-1 path
loadAndExportJson's directory failures already used (export has never used
exit 2 for a metadata problem, only for a bad CLI flag), so the existing
back-compat contract holds exactly.
- index.ts — the no-args "is this a MetaObjects project?" probe, previously
the one site that inlined the "metaobjects" string literal instead of
importing the constant.
init.ts is untouched (it's the scaffolder writing the default) and so is
detect-stack.ts (separate task).
New collection-routing.test.ts proves a project with `sources` declared
elsewhere generates successfully with no metaobjects/ directory anywhere.
Rebuilt packages/sdk's stale dist/ (missing resolveCollection and
loadMemory's `files` option) so the dist-based gen-split-tree regression
gate could resolve them.
cli: 557 pass / 3 skip / 0 fail (556 pre-existing + 1 new). codegen-ts
golden-output gate: 1241 pass / 0 fail, unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gen-split-tree test gate runs the built CLI under node, requiring all runtime imports to have dist/ at least as new as src/. The helper ensureFreshDist() checked codegen-ts and cli but not sdk, leaving a stale-dist hole: any change to sdk/src/ was invisible to the gate until dist/ was rebuilt by hand. Add sdk to the rebuild loop following the exact pattern of codegen-ts: resolve the package root via createRequire, then check srcDir and distFile with the stale-rebuild behavior already wired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ymlink blindness hasRequirementNodes now scans (await resolveCollection(cwd)).files instead of a bespoke readdirSync walk hardcoded to metaobjects/. This fixes two bugs at once: a project that declares `sources` elsewhere had its requirement.* nodes silently invisible to agent-context scaffolding, and the old walk used Dirent.isDirectory() (false for a symlinked directory) so it never descended into a nested symlinked subdirectory — while the loader (stat-based) does. Both the bespoke walk and the METADATA_DIR constant are deleted. resolveStack and probe are now async; init.ts's stackForAgentContext caller chain is updated to await. meta init still succeeds on an empty directory — resolveCollection's ERR_COLLECTION_NOT_FOUND (no metadata yet) is caught and treated as "no requirement nodes", preserving the heuristic's never-throws contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Array literal inferring element type as string instead of literal union, causing TS2769 when passed to ERROR_CODES.toContain(). Preserving literal types with as const keeps the type-safe comparison. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…er touched
`meta migrate` and `meta verify --db` now route through `resolveCollection()` and
honour the per-command `migrate.scope` it carries. A consumer sharing a database
with another owner declares:
"migrate": { "scope": ["acme::platform::**"] }
and every table or view whose DECLARING object falls outside is neither created,
altered, dropped, nor reported as drift.
The suppression is two-sided, deliberately. Dropping out-of-scope tables from the
expected schema alone is strictly worse than not having the feature: `diff`
proposes a DROP for everything present in `actual` and absent from `expected`, so
expected-only filtering converts every out-of-scope table that EXISTS in the
database into a proposed `DROP TABLE` — the precise hazard the feature exists to
remove. The out-of-scope names therefore also ride `diff`'s `unmanagedNames` seam
(merged with `collectUnmanagedNames`, never replacing it), the same mechanism
`@unmanaged` uses to stop an external table being dropped.
The scope decision is made on the declaring object's `resolutionKey()`, threaded
out of the pass that already holds it (`buildExpectedSchemaWithProvenance`;
`buildExpectedSchema` is now a thin wrapper over it). It is NOT re-derived from
the SQL name, which is lossy, and NOT collected by a second metadata walk, which
would have to duplicate Pass 1's skip rules — abstract / TPH subtype / no
writable source / `@unmanaged` — and would drift from them. A TPH table is
attributed to its discriminator BASE, so an out-of-scope subtype can never
suppress the base's table.
Provenance never reaches disk: `SNAPSHOT_FORMAT_VERSION` stays 3 and an unscoped
project's snapshot bytes are unchanged (`canonicalize()` spreads the descriptor,
so a descriptor field would land in the committed snapshot and owe a format bump
that hard-fails older readers). View provenance travels on `ExpectedViewInput.fqn`
and is recorded in the map only.
Covered paths: the online Kysely diff, the offline snapshot diff (`planOffline`),
the D1 diff, and `computeDriftFromActual` — the single choke point both
`verify --db` paths share. `verify --db` prints one line naming what it excluded
(silence would misreport an unchecked table as a checked one), and its committed-
snapshot check (#292) filters out-of-scope objects from BOTH sides, since a scoped
migrate writes a scoped snapshot. `migrate baseline` is deliberately unscoped: the
`--from-db` arm captures whatever the database holds, and an offline baseline that
recorded less would disagree with it.
Unchanged by design: a table NO loaded object declares is still a proposed drop —
scope only silences objects that were loaded and fell outside it. A project with
no `migrate.scope` gets a byte-identical migration, a byte-identical snapshot and
an unchanged drift verdict (verified by regenerating a fixed project's snapshot +
up/down.sql before and after: all three md5s identical).
One qualified-name definition now serves all three keyers (`diff`'s identity maps,
`collectUnmanagedNames`, the out-of-scope set) — a second spelling would silently
un-suppress an object and propose its drop.
migrate-ts: 753 pass / 22 skip / 0 fail. cli: 568 pass / 3 skip / 0 fail.
codegen-ts golden gate: 1241 pass / 0 fail. Workspace typecheck: 18/18 exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…--codegen honours it `runGen` gains an optional `scope?: (fqn: string) => boolean` predicate applied at its single entity-selection choke point, intersecting with `entityFilter` and matched against `obj.resolutionKey()` — a plain predicate, not the sdk-owned pattern strings, since codegen-ts must not depend on @metaobjectsdev/sdk. `meta gen` now always passes `(fqn) => matchesScope(fqn, collection.scope)`; an unconfigured project's compiled scope has an empty include/exclude, which `matchesScope` treats as "everything", so this is a no-op for the common case. `verify --codegen` threads the identical predicate into `computeCodegenDrift`'s regeneration pass (design §7 open question 3) — otherwise a scoped `meta gen` and an unscoped regen disagree about which files should exist, and every out-of-scope entity reads as false drift. `runGen`'s "No entities to generate" warning now distinguishes a scope that admitted nothing from an empty root or an unmatched entityFilter, rather than misattributing the cause. The scope===undefined path is byte-identical to the pre-scope two-way branch, proven by the golden-output gate (unmoved, 1241 pre-existing assertions untouched) plus a new content-comparison test. Dangling references (an in-scope object referencing an out-of-scope FK target / @objectref / projection base) are documented, not warned on: detecting them correctly needs a general reference-walker across every reference kind, which is new machinery this task's seam doesn't fit: the closest existing map (relation-resolver's targetEntity) is bare-name only (the #228 limitation), and the actual failure mode is a loud compiler error on the generated code's unresolved import, not a silent one. codegen-ts: 1245 pass / 0 fail (1241 baseline + 4 new, golden gate unmoved). cli: 571 pass / 3 skip / 0 fail (568 baseline + 3 new). Workspace typecheck: 18/18 exit 0 — caught a real implicit-any in verify.ts's pre-existing `let collection;` once it was read from a nested function, fixed with an explicit type annotation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tree resolveCollection + loadMemory + matchesScope run against examples/advanced-modeling/metaobjects/ (acme::learn, 3 files, 9 objects) — zero new metadata authored. Three angles: a synthetic consumer reaching the tree via an absolute-path source; the examples project's own committed config (sources: [] falling back to metaobjects/, the exact shape every `meta init` scaffold produces); and scope patterns (**, single-segment *, Program*-exclude) evaluated over the real resolutionKey() FQNs the loader produced, not hardcoded object names.
Nine existing cases never exercised case — a port could implement case-insensitive matching and pass the corpus clean, the same shape as the cross-port LIKE/ILIKE divergence this corpus's own README cites as its reason for existing. Adds matching-is-case-sensitive plus the corresponding README Semantics bullet. No product-code change: compilePattern already builds its RegExp with no `i` flag, verified against the case before adding it to the corpus.
Adopter guide for phase-1 metadata source resolution, describing what shipped rather than what was planned: - docs/features/metadata-sources.md (new) — `sources` as a set with `metaobjects/` as its DEFAULT value (so `"sources": []`, which `meta init` has always scaffolded, changes nothing); the `path` kind, read in place and never installed; the `*`/`**` pattern grammar; nearest-ancestor discovery and the `.git` boundary; `migrate.scope` and why a `migrate` block belongs where the ledger lives; vendoring as "copy a directory and point a `path` at it"; a worked polyglot example (a Maven module owning the model, two Node consumers, one schema owner). - cli/README.md — `sources`, `scope` and `migrate.scope` documented in the config reference beside `targets`. - CLAUDE.md — two additive lines: `metaobjects/` is a default, never a requirement; metadata location resolves via `resolveCollection()`, and only `meta init` may hardcode the directory name. - docs/CONFORMANCE.md — `fixtures/scope-conformance/` (10 cases) added to the corpus matrix and given a detail subsection; total 19 -> 20. TS is the only port with a runner today; the other four are deferred to the ports plan. - design doc §3 — the sharpened three-layer order-independence statement (resolveSources canonicalizes; the loader resolves content order-free; sibling order of unrelated top-level nodes is NOT a contract), naming the gate that pins layers 1 and 2. §4.7's whole-tree byte-identity bar is marked corrected so it stops contradicting §3. Deliberately documents only shipped behaviour: `resource`/`package` sources, per-generator declarative scope, and the non-TS CLIs are listed as deferred.
`resolveSources` returned a flat lexicographic sort of absolute paths, while every read path before this branch walked metadata through `listMetadataFiles` — the files at a level, then that level's subdirectories, depth-first. The two disagree the moment a subdirectory name sorts before a sibling file (`metaobjects/common/…` ahead of `metaobjects/meta.users.json`), and that order is observable in output: the barrel generator emits exports straight from `root.objects()` order, and the same order flows into the shared `enums.ts`, `meta docs` page ordering and `meta export`'s sibling order. A project with a nested metadata directory therefore got REORDERED generated code — against the absolute promise that a project declaring no `sources` and no `scope` is byte-identical to before. There is now exactly ONE walker: `listMetadataFiles` is exported and `resolveSources` calls it rather than keeping a second recursive walk of its own (the two had already drifted once, on case-sensitivity). Across specs the order is decided by spec CONTENT — the specs are walked in `JSON.stringify` order, not declared order — so the full result stays a pure function of the declared SET, and `order-independence.test.ts` passes unchanged. Every `metaobjects/` tree committed in this repository is flat, so nothing here could observe the property. `sdk/test/source-order.test.ts` builds the shape that discriminates and pins it four ways: the explicit expected order, a check that the fixture would sort the other way (so the gate cannot go vacuous), equality with `listMetadataFiles` on the same tree, and the loaded tree's sibling order. `dogfood-examples.test.ts` now asserts the committed examples tree resolves in that same walk order instead of merely "sorted", and takes its directory name from `DEFAULT_METADATA_DIR` rather than the literal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…P TABLE Narrowing the expected side could WIDEN what the diff governs. `diff` derives its SCHEMA scope from the schemas the expected side mentions and falls back to "no schema scoping at all" when expected is empty (the legacy whole-database path for a project with no model). A `migrate.scope` matching nothing empties `expected`, reaches that fallback, and every actual table in every schema becomes a drop candidate — another owner's included, which was never in `expected` so it carries no provenance and never lands in `outOfScope` either. So the declaration whose entire purpose is to stop migrate touching another owner's tables CAUSED migrate to propose dropping one, silently, whenever the scope was wrong. The same mechanism reached `verify --db` as phantom drift. Structural fix: `scopeExpectedSchema` reports the UNSCOPED model's schemas as `declaredSchemas`, and every narrowing caller threads it into `diff`'s existing `scopeSchemas` — `planOffline`, `computeDriftFromActual`, both CLI migrate paths, and verify's committed-snapshot gate (which narrows the snapshot by the same `outOfScope` set and had the identical hole). The schema scope is now a property of the whole model, which `migrate.scope` cannot move in either direction. An UNSCOPED project passes no `scopeSchemas` at all, exactly as before — `declaredSchemas` is undefined without a predicate, so its arguments to `diff` are unchanged. Defensive fix: a `migrate.scope` matching zero loaded objects is refused, on all four command paths, naming the patterns that missed and the FQNs that were loaded. It can never be what someone meant, and left alone it is silent — migrate reports "no changes" having compared nothing. `Collection` carries `migrateScopePatterns` so the message can show what the author wrote rather than a compiled regex source. Refused inside verify's schema gate rather than beside its collection load, so a stale pattern cannot fail a `--templates` run that never consults it. Gated by `migrate-ts/test/scope-empty-match.test.ts` (the reviewer's probe: control drops nothing, empty-match scope must too) and `cli/test/integration/migrate-db-scope.test.ts` on the online `--db` path, which had no scope test at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…fail-open config blocks
I3 — `meta prompt-snapshot` was the one live read path still deciding for itself
where metadata lives: `loadMemory(cwd)` with no `files`, plus the ENOENT sniff
every other site removed. `--check` is a drift GATE, so a project declaring
`sources` elsewhere either got "no metaobjects/ found" or, with a stale
`metaobjects/` still on disk, gated silently against the wrong model. Routed
exactly as `gen.ts` does, and everything project-relative (the `.metaobjects/
snapshots/` goldens, the `prompts/` a `@textRef` resolves in) now hangs off the
resolved config dir rather than ambient cwd, so the metadata and the text it
names can never come from two different projects.
This was a spec gap, not a missed task: design §4.6.0's site table enumerated
nine reads and omitted this one, which is why nothing scheduled it. The row is
added, with a note saying why — the ports plan is written from that table and
would otherwise inherit the omission in four more languages.
I4 — `MigrateBlock` and `D1Block` were `.partial()` under zod's default STRIP
policy, so `{ migrate: { scopee: [...], dialect: "postgres" } }` parsed to
`{ dialect: "postgres" }`: a typo'd `scope` key silently meant UNSCOPED, which
is the "migrate.scope matched nothing" hazard through a second door. Both are
`.strict()` now, matching the top level, the `sources` arms and `ScopeSchema`,
each of which already carried a comment about this exact fail-open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…mbient cwd I5 — metadata resolved through `resolveCollection` (nearest ancestor holding `.metaobjects/config.json`) while `metaobjects.config.ts` and the `.metaobjects/config.json` operational block were still read from cwd, at six sites. Run `meta migrate` from a subdirectory of a project whose root declares `columnNamingStrategy: "literal"`: the metadata comes from the ancestor, the strategy silently defaults to `snake_case`, and the emitted migration RENAMES EVERY COLUMN. Newly reachable — pre-branch that invocation failed outright with "no metaobjects/ found" — and it aligns the code with design §4.6.1, which already said per-port generator config is read from that same directory. The line drawn, and it is the same in all three commands: anything named BY the metadata or its config resolves against the config dir — `metaobjects.config.ts`, `.metaobjects/config.json`, the `outDir` and `wranglerConfigPath` they carry, the `prompts/` a `@textRef` resolves in, the test files a `@verifiedBy` names. Anything that is merely "the tree the user is standing in" stays on cwd: the agent-context staleness nudge and the advisory anti-pattern scan, both warnings-only. For a run from the project root every path is identical, which is the only invocation that worked before. The relative-path interaction the fix had to decide: a relative `outDir` resolved against the resolved cwd (a deliberate fix in 0.19.2). It now resolves against the config dir, because the config it comes from does — otherwise a subdirectory run writes its migration where the next run cannot find it. `migrate` computes its root with `findConfigDir` rather than `resolveCollection`, deliberately: `apply-pending` and `--rollback` replay committed SQL and load no metadata at all, so requiring metadata to exist would be a regression. It falls back to cwd exactly as `resolveCollection` does, so the two agree by construction. Gated by `cli/test/integration/migrate-config-dir.test.ts`: a subdirectory run must emit BYTE-IDENTICAL SQL to a project-root run, and must write under the project root. Both fail on the pre-fix `metaRoot = cwd`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
I6 — `loadMemory(configDir, { files })` bypasses `collectMetadataPaths`'s
`package.meta.json` peer-package walk entirely. Design §11 rules that
intentional and it fails loudly (`ERR_UNRESOLVED_SUPER`) rather than silently,
but the adopter guide did not mention `package.meta.json` or workspaces
anywhere — so the one mechanism a project could be relying on was documented
nowhere, including its replacement.
Documented in an Upgrading section together with the other two changes an
existing project can notice: `ConfigSchema` is now `.strict()` (a previously
stripped key is a load error), and `ExpectedView.fqn` became required on a
public `codegen-ts` export. Mirrored under the existing `## [Unreleased]`
heading in CHANGELOG.md.
Also corrects the two places the guide described `resolveSources` as sorting
absolute paths, which stopped being true when the resolver was fixed to keep
the loader's per-level walk order.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1. `fixtures/scope-conformance/README.md` named `isInScope(...)`; the symbol is `matchesScope`. Four ports implement from that file. 2. `logOutOfScope` used `log.info` unconditionally, so a scoped `meta migrate --format json` put a prose line on stdout ahead of the JSON and broke `| jq`. Text format keeps stdout; every other format routes it to stderr rather than dropping it — the non-TTY default format is toon, so suppressing outright would silence the note for every piped and CI run. `verify` is NOT changed: it never receives `fmt` (index.ts passes it only to `gen` and `migrate`) and emits prose throughout, so there is no structured document to interleave with. 3. `gen-scope.test.ts`'s "byte-identical to today" case asserted three FILENAMES and read no bytes — a title claiming a byte guarantee over a test that cannot see one is how the real guarantee went unexamined. It now compares an unscoped run against an all-matching scope file-for-file, byte-for-byte, and is titled for what it checks. 4. (Shipped with the C1 commit) `dogfood-examples.test.ts` builds its metadata dir from `DEFAULT_METADATA_DIR`, not the literal. 5. CLAUDE.md claimed `meta init` was the sanctioned exception to hardcoding the directory name; `init.ts` has always IMPORTED the constant. The only literal is `sdk/src/memory.ts:18`, which is now what the sentence says. 6. "the phase-1 ports plan" was cited three times as if it named a findable artifact; no such document exists. Reworded to describe the future work. 7. An accepted scoped run persisted the NARROWED schema as the committed snapshot, deleting every out-of-scope entry — so later widening or removing `migrate.scope` proposed CREATE TABLE for a table that exists and failed at apply. `carryForwardOutOfScope` keeps those entries, on both the offline and the `--from-db` paths. `PlanOfflineResult` now separates `nextSnapshot` (what to COMMIT) from `expected` (the governed side the diff compared and the emitter renders against), so the emitter's input is unchanged. Unscoped runs commit the same object, so the snapshot stays byte-identical. The `migrate baseline` comment that claimed an out-of-scope entry in the snapshot was harmless is now true, and says why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Self-review follow-up to the C2 fix: `declaredSchemasOf(snapshot)` was called twice in the same expression — once to decide whether to pass `scopeSchemas`, once to build it. One binding, named for what it is, with the reasoning hoisted above the `diff` call where a reader meets it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Part A of the consolidated quality pass over the metadata-source-resolution branch. Every item is a guarantee the toolchain already made and did not keep. A1 — `meta docs --site` hard-failed on the flagship multi-source config. The CLI's dedup set was SEEDED from the source dirs, so it only ever guarded the `templates/` additions, and two declared sources sharing a basename (`metaobjects` plus `../shared-model/metaobjects`) reached the site loader as a "duplicate source dir basename" error. The site keys its source groups by that basename, so the fix is in the loader: collisions are qualified by their parent directory, then by a counter, and the returned group names are the staging names `treeOf` actually matches. The CLI dedups by resolved PATH — two different dirs sharing a name are legitimate; the same dir twice is the real hazard. A2 — discovery walked past a LOCAL `metaobjects/`. A nested project holding its own metadata and no config of its own read that metadata before this branch; the config-only stop condition made it silently load the ANCESTOR's model and write generated output to the ancestor's outDir. The walk now stops at the first ancestor carrying EITHER `.metaobjects/config.json` OR a `metaobjects/` directory, config checked first. Stopping on the latter means the default sources apply, which is exactly the pre-branch behaviour for that directory. Design §4.6.1 amended. `findConfigDir` is replaced by `discoverCollectionRoot` (dir + hasConfig) and `resolveConfigDir`, which `migrate` now calls — two walks with different stop conditions is the drift this branch exists to remove. A3 — `meta docs` never adopted `projectRoot`. It read `metaobjects.config.ts`, the docs `outDir`, `templates/` and the owned `codegen/docs-site/` theme from the ambient `<metadata>` argument, resolving the collection only afterwards, so a run from a subdirectory rendered the ancestor's metadata with the subdirectory's absent providers. Discovery now runs first and `projectRoot = collection.configDir`, matching `gen`, `verify` and `prompt-snapshot`. `--scaffold-site` resolves the same root, so it writes where `--site` reads. A4 — `meta export`'s two adopter-visible changes (files-before-subdirectories sibling order, `_pending/` excluded) are documented in the Upgrading section and the changelog. Both are correct; neither was written down. A5 — the last whole-database door. The committed-snapshot gate guarded on `snapshotSchemas.length > 0`, so a snapshot that EXISTS but is empty combined with a non-empty out-of-scope set still reached `diff`'s "no model, govern the whole database" fallback: a third party's table, in a schema this model never declares, became a drop candidate reported as a snapshot disagreement. Closed through the shared helper rather than beside it — the gate now takes the scope decision the drift comparison already made (`DriftResult` reports its `declaredSchemas`) instead of re-deriving a pin from the snapshot. A6 — the previous brief's premise was wrong in the other direction, so the real semantics is pinned by test and stated in `scope.ts`'s header: a scope narrows which OBJECTS the tool governs, never which SCHEMAS it may see. A schema whose every declared object is excluded stays in scope, so another owner's undeclared table in it is still a drop candidate — the same verdict an unscoped run gives. Deriving the schema set from the survivors instead reintroduces the inversion. A7 — the migrations directory follows the project root, which moves the ledger for the two subcommands that load no metadata (`apply-pending`, `--rollback`). Kept, because the ledger belongs with the config that declares it, but made loud: `migrate` names the directory it is using whenever it differs from `<cwd>/.metaobjects/migrations` and that local directory exists. A8 — `gen` and `verify` describe the anti-pattern scan and the agent-context nudge as the same advisory pass and scanned two different trees for it. Both now root at `projectRoot`; a `verify` run from a subdirectory previously found no agent-context manifest at all, so the nudge silently never fired. A9 — one comment naming the carry-forward trade-off: the entries carried into a scoped run's committed snapshot are INTROSPECTED descriptors, so removing the scope later can produce one round of cosmetic alter churn. Strictly better than the `CREATE TABLE`-on-an-existing-table it replaced, and identical in kind to `baseline --from-db`. The shared `scopedDiffInputs` / `excludeFromSnapshot` helper (Part B's highest- value finding) lands here because A5 cannot be fixed correctly without it: the three-part scoped-diff contract was enforced by prose at five call sites, and site five had already drifted into its own guard. All five now compose through one door. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Part B of the quality pass. Every item here is a rule that was stated in prose and re-derived at N call sites — the shape whose failure at site N+1 is this branch's entire premise. (B1 and B3 shipped with Part A: A5 could not be fixed without B1's shared helper, and A2's stop-condition change forced B3's single discovery definition.) B2 — `Collection` exposed a `CompiledScope` and every consumer immediately wrapped it in the same lambda; nothing ever consumed one as a compiled scope, which is why `migrateScopePatterns` had to exist alongside it (a compiled scope cannot be shown to a human). It now carries predicates: `inScope`, always defined so callers pass it through without branching, and `inMigrateScope`, whose undefined-ness is load-bearing — that is what leaves an unscoped run's expected schema untouched. `toObjectScope` is deleted; `compileScope`/`matchesScope` stay exported for the conformance corpus. The window was now: `Collection` is new on this branch. B4 — the scope-mismatch refusal was three byte-identical copies differing only in a local variable name. One `refuseScopeMismatch`, so the hint string and the exit code are one decision. B5 — `docs.ts` resolved the collection twice on the `--model --site` path, and re-derived spec→path resolution byte-for-byte from `sources.ts`. `emitSite` now takes the resolved collection (and loses its own try/catch), and the source roots come from the collection, which derives them from the DECLARED specs in `resolveSources`'s own canonical order. That last part is a fix, not a tidy: dirs were derived from resolved FILES, so a declared source directory holding no metadata vanished from the site's group list entirely and `sourceDirs` could come back empty where the pre-branch code always passed `<root>/metaobjects`. B6 — `assertPathSpec` was called twice per spec, the second call carrying a comment admitting it existed to re-narrow for TypeScript. A returning `toPathSpec` does both jobs once, and the validate-then-order pass it feeds is now a named `orderedPathSpecs` — exported, because `sourceRoots` must use the identical canonical order and a second sort would be a second definition of "canonical". B7 — the three-level nested ternary in `runner.ts` is an if/else chain. Behaviour-preserving, quirk included: the unscoped arm still blames `entityFilter` for an empty root, and the warning strings are untouched. B8 — `loadMemory`'s `repoRoot` is inert whenever `files` is supplied, which is all eight routed call sites. Documented rather than re-signed: a ninth call site copying the shape but forgetting `files` silently loads `<repoRoot>/metaobjects/`, which is the divergence this design closes. C2 lands here too — `outOfScopeNote` is the same sentence `migrate` and `verify` built separately, and both were already being rewritten for B2/B4. Byte-identical output for both commands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Part C. `migrate-db-scope` and `verify-db-scope` scaffold the identical two-owner project — the same `PLATFORM` metadata, the same `declareScope`, a near-identical `scaffold` — because `migrate` and `verify --db` govern the identical object set from ONE declaration. Two copies of that project had already drifted: `ARENA` was a constant in one file and a `(venue: boolean)` factory in the other, which is how two suites meant to prove the same contract quietly stop testing the same thing. `test/integration/support/scope-fixture.ts` exports `PLATFORM`, `arena(opts)`, `arenaFile`, `scaffold(prefix)` and `declareScope`. The `prefix` argument keeps each suite's temp directories self-identifying. The `console.log` capture boilerplate stays where it is — it is a pre-existing convention across fifteen CLI test files, and consolidating it is a different change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Self-review of the A3 change. `--templates` redirects the adopter RENDER template chain, and routing the owned `codegen/docs-site/` theme through the same `projectRoot` would have made `meta docs --site --templates <dir>` look for a theme in a directory `--scaffold-site` never writes to. Both now key on `collection.configDir`, so the two halves of scaffold-and-own cannot separate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…oader entry Two sites still answered "where does metadata live?" without reading the config. Both are read paths, so both could silently load from somewhere the project never declared. 1 — discovery stops at `.metaobjects/config.json` and nothing else. A quality-pass ruling had added a second stop marker: a bare `metaobjects/` directory also ended the walk, to keep a nested config-less project reading its own metadata. That is withdrawn. `metaobjects/` is the DEFAULT VALUE of `sources` and carries no other meaning, so a directory of that name says nothing about whether a project lives there — and a second marker is a second definition of where metadata lives, which is precisely the duplication `resolveCollection` exists to be the only instance of. It was also wrong on its own terms: a project pointing `sources` at a sibling module has no such directory, and one with both would be governed by whichever the walk noticed first. The consequence is real and is documented as the rule rather than as a caveat: a subdirectory holding metadata but no config now resolves to its nearest ancestor config. A subdirectory that should own its metadata declares one — `meta init` writes it, and `"sources": []` is enough to claim the directory. 2 — `loadMemory` resolves through `resolveCollection` too. Its no-`files` arm scanned `<repoRoot>/<default dir>` directly and walked `package.meta.json` workspace peers, so a caller that copied the routed shape but forgot `files` loaded from a directory the config may never have mentioned — the ninth call site, arriving by omission. Both arms are now the same answer: one already computed by the caller, one computed here. The peer walk goes with it (design §11 pre-ruled the retirement; the Upgrading section documents it). The constants and the metadata-file walk move to a new leaf module, `metadata-files.ts`, which imports nothing from its siblings. Homing them in `memory.ts` closes an ESM cycle whose failure mode is a crash, not a warning: `DEFAULT_SOURCES` reads `DEFAULT_METADATA_DIR` at module top level, so the cycle surfaces as `ReferenceError: Cannot access 'DEFAULT_METADATA_DIR' before initialization`. A lazy `await import()` would hide that rather than remove it. Tests: two discovery cases retargeted to assert the withdrawn behaviour's absence, one dropped (it only guarded that marker); `resolveCollection`'s back-compat case inverted to assert the ancestor config governs. The two retired-peer-walk cases now pin what the Upgrading section PROMISES about the removal — that it fails loudly with `ERR_UNRESOLVED_SUPER`, and that a declared source replaces it — rather than being deleted along with the feature. `loadMemory` with nothing to resolve now reports `ERR_COLLECTION_NOT_FOUND`, the same structured code every other command gives. sdk 243 -> 242 (one test dropped, six retargeted); cli, migrate-ts and codegen-ts unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… may not have The no-args probe already routed through `resolveCollection`, and the comment above it said so — then the next three lines printed `metaobjects/ found`, `no metaobjects/ here`, and `Scaffold metaobjects/ in this directory`. For a project whose config points `sources` at a sibling module the first is simply false, and the second blames a missing default directory when the real problem may be a declared source that failed to resolve. The status line now reports what the probe actually established — metadata found, or no MetaObjects project here — and the next step offers to scaffold a project rather than a named directory. Swept the rest of `cli/src` for user-facing strings making the same claim: `gen`'s two help summaries, the `<metadata>` positional in both docs help slices, the `--prompts` note, and `gen`'s empty-result hint. `meta init`'s own output keeps naming the directory — it CREATES it, which is the one place the name belongs. Comments are untouched. Two tests pin the rule at the seam it broke: a project whose sources point elsewhere, and a directory with no project at all, each asserting the printed status contains no directory name. Both were confirmed red against the previous strings. The committed help snapshot caught the help-text edits, as intended, and is regenerated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`metaobjects/` is the DEFAULT VALUE of `sources` and nothing else. That was implicit in the design and contradicted by three code paths; with those fixed, write it down where someone will hit it before repeating them. - CLAUDE.md names the four sites that may spell the directory — the constant's definition, `DEFAULT_SOURCES`, `resolveCollection` applying it, and `meta init` writing the layout — and points at the test that enforces the list. - `metadata-sources.md` states the rule positively in the body rather than as a footnote: a project that declares `sources` may put metadata anywhere and need not have a directory of that name at all, and every command follows the config together. - Upgrading gains the consequence of the discovery change, phrased as the rule: a project boundary is a `.metaobjects/config.json`, so a subdirectory holding metadata but no config resolves to its nearest ancestor config. If it should own its metadata, give it one. - The retired-workspace-walk entry said "every CLI read path"; `loadMemory` itself now resolves the same way, so it says every read path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…spirational
Eight call sites once hardcoded the metadata directory. Routing them through
`resolveCollection` fixed those eight and nothing about the ninth. This is the
part that lasts: a test walking `sdk/src` and `cli/src` that fails when a file
outside an allowlist names the directory in code.
Three things keep it from being a gate that passes because it checks nothing:
the allowlist is file + REASON, so a new entry costs a written justification; a
STALE entry fails, because an allowlisted file that no longer holds the
reference is an unwatched exemption the next file inherits; and comments are
excluded, proven against `detect-stack.ts`, which mentions the directory only
in a comment saying it does not assume it.
Both failure modes were demonstrated rather than assumed. A temporary
`join("x", "metaobjects")` in `discovery.ts` produced
`["sdk/src/discovery.ts:113"]`; a temporary allowlist entry for a file with no
reference produced `["sdk/src/scope.ts"]`. Both were then removed.
Writing the guard found what a grep did not. Three lines were the PRODUCT name
in prose — "the metaobjects ledger", "reach for metaobjects metadata", the
`metaobjects:` error prefix — so a reference now requires a following `/` or
closing quote. That is the guard's sharpest limit and its limits are recorded
in the file header, measured row by row: it catches plain literals, template
literals and messages with a trailing slash; it misses any computed spelling
and the bare word without a slash.
Two consequences beyond the four sanctioned sites:
- `cli/src/index.ts`'s two `init` help lines named the directory to everyone,
in every project, including one whose sources point elsewhere. Reworded —
`meta init`'s OWN output still announces the layout it writes, which is where
that belongs.
- `sdk/src/agent-docs/body.ts` is allowlisted with its reason: it is the
scaffolded documentation prose, reachable by no read path, teaching the
default layout a fresh project gets. Flagged as a wording gap for a project
that declares `sources` elsewhere, not a resolution one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 — `runGen`'s `scope?` JSDoc told callers to adapt a compiled scope with `(fqn) => matchesScope(fqn, collection.scope)`. There is no `collection.scope`: a `Collection` exposes the predicate directly as `inScope`, and the only caller passes it straight through. The paragraph's substantive point — do not turn this seam into a config surface — still stands; only its closing sentence was false. Same stale expression in a comment in `run-gen.test.ts`. 2 — `warnIfLedgerRelocated` compared against `resolvePath(metaRoot, config.outDir)`, but under `--migration-format flyway` with a default outDir the directory the run WRITES to comes from `resolveFormatOutDir`, which redirects to `src/main/resources/db/migration`. In that combination the warning named a directory the invocation would never touch — a message whose entire job is to say which directory is in use. It now takes the resolved format directory. The warning was covered by nothing at all, which is why it could be wrong. A test now runs the exact layout it was written for — a subdirectory holding its own ledger, under a project root that declares the config — and asserts the warning names the flyway directory and not the default one. Confirmed red against the previous call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four accuracy defects (prose disagreeing with the code it describes) and
one genuine behaviour fix, per the source-resolution rule: metaobjects/
is a config default, resolved everywhere through resolveCollection.
- CLAUDE.md: the rule's own allowlist citation said "four sites" against
a six-entry allowlist (sdk/src/index.ts and agent-docs/body.ts were
added since); named all six.
- docs/features/metadata-sources.md: "nothing tests for its existence or
mentions it in a message" was false on both counts; the Upgrading
intro miscounted its own subsections ("Seven" over six); loadMemory's
no-files rejection silently changed from a plain Error to a
ParseError/ERR_COLLECTION_NOT_FOUND and that break was undocumented.
- sdk/test/discovery.test.ts: a comment still described the withdrawn
"bare metaobjects/ directory is a second stop marker" behaviour that
the describe block below it now asserts is NOT the case.
- sdk/test/no-hardcoded-metadata-dir.test.ts: recorded one more blind
spot in the guard's comment-stripper — a regex literal containing `//`
is misread as a line comment, blanking a violation to its right.
- cli/src/commands/migrate.ts: the relocated-ledger warning used the
Kysely-path directory convention for a plain `--dialect d1` run, so it
could name a directory that run never writes to. Fixed rather than
just re-documented: D1's own convention is now a shared helper
(resolveD1OutDir) called once the wrangler binding resolves, from
inside runD1Migrate, instead of the generic pre-dispatch check guessing
at it. The `--migration-format flyway` + `--dialect d1` combination
(refused before either directory is touched) keeps using the generic
check, unchanged.
sdk 247/0, cli 580/3 skip/0 (+1 test for the migrate.ts fix); full
workspace typecheck green across all 18 packages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… provenance refusal
…/ is default location
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Phase 1 of metadata source resolution (TypeScript). Replaces a hardcoded
metaobjects/directory read independently by eight call sites with ONE authority —resolveCollection()in @metaobjectsdev/sdk — and adds three config keys in .metaobjects/config.json:sources(a SET of source specs; only thepathkind resolves in phase 1, whileresourceandpackageare registered and throw ERR_SOURCE_KIND_UNSUPPORTED),scope(include/exclude package patterns over fully-qualified names, applied at OUTPUT only — it filtersmeta gen, andmeta verify --codegenhonours the same scope so it cannot report false drift), andmigrate.scope(governsmeta migrateandmeta verify --db; out-of-scope tables and views are removed from the expected schema AND suppressed on the actual side through the existing unmanagedNames seam in diff(), so they are neither created nor dropped). Also: nearest-ancestor config discovery bounded by .git; four new cross-port error codes registered in all five language ports; a new fixtures/scope-conformance/ corpus of 10 cases plus an order-independence gate; and an adopter guide with an Upgrading section.THE GOVERNING INVARIANT, stated by the project owner mid-review:
metaobjects/is the DEFAULT VALUE ofsourcesand nothing else. Exactly six sanctioned sites may name that directory — the constant definition, DEFAULT_SOURCES, the default-applies check inside resolveCollection, the sdk barrel re-export, themeta initscaffolder (which CREATES it rather than assuming it), and the agent-docs prose that init scaffolds. Every other path reads the config. That rule is enforced by a guard test carrying a file-plus-reason allowlist which also fails when an allowlist entry goes stale. An earlier controller ruling that had discovery stop on a baremetaobjects/directory was withdrawn under this rule and reverted deliberately: a project boundary is a .metaobjects/config.json, and a bare directory is not a project.BACKWARD COMPATIBILITY was the hardest constraint and is absolute: a project with one config at its root, no
sourcesand noscope, must produce byte-identical generated code, migrations and committed schema snapshot. SNAPSHOT_FORMAT_VERSION deliberately stays 3 — adding a field to a schema descriptor would have forced a format bump that hard-fails older readers, which a scoping feature must not cost adopters.DELIBERATE SCOPE DECISIONS a reviewer reading only the diff would not know:
meta docsandmeta exportare intentionally NOT scoped, being inspection surfaces over the loaded collection rather than code emitters; per-generatorscopeis not phase 1 and the existing TypeScript-only per-generatorfilterfunction stays unchanged as an escape hatch;meta migrate baselineis deliberately unscoped because a --from-db baseline has no provenance to scope by; and theexplicitDiroption on resolveCollection is retained despite having no production caller because it is specified phase-1 surface. The codegen scope seam takes a predicate rather than pattern strings because codegen-ts must not depend on sdk — the rule that scope is package patterns and never a predicate function governs CONFIG SURFACES that must port to a pom.xml and a YAML config, not internal plumbing between two TypeScript packages.ADOPTER-VISIBLE CHANGES are documented in the Upgrading section of docs/features/metadata-sources.md and mirrored in CHANGELOG [Unreleased]: the workspace package.meta.json
extends:peer-discovery walk is retired, ConfigSchema is now .strict(), ExpectedView.fqn became required,meta exportoutput order changed and it now skips _pending/, and loadMemory(dir) called without an explicit file set now rejects with ERR_COLLECTION_NOT_FOUND instead of a plain Error.PROCESS: executed via superpowers:subagent-driven-development against docs/superpowers/plans/2026-08-17-metadata-source-resolution-phase1-ts.md — 15 planned tasks plus one controller-authored task, each individually reviewed, then a whole-branch review (2 Critical, 4 Important, all fixed and re-reviewed clean), a code-review high pass (4 findings), a code-simplifier pass (12 findings plus 11 explicit non-findings), and a 19-item consolidated fix. The two Criticals are worth naming because both were invisible to the existing gates for structural reasons. FIRST: resolved sources were flat-sorted by absolute path while the walker used by the loader sorted files within each level and visited subdirectories after them, so a nested metadata layout produced a different object order and therefore different generated bytes — every metaobjects/ tree in this repository is flat, so the golden-output gate was structurally blind to it. SECOND: a migrate.scope matching nothing emptied the expected schema, which made diff fall back to its legacy whole-database path with no schema scoping and propose DROP TABLE against another owner schema — every scope test on the branch used a scope that matched something. Both now ship with the regression check that was missing, and the second fix pins the scopeSchemas argument of diff to the UNSCOPED model so narrowing can never widen the comparison.
What Changed
resolveCollection()in @metaobjectsdev/sdk, replacing eight hardcodedmetaobjects/directory reads across CLI commands and migrate toolingsources,scope, andmigrate.scopekeys to .metaobjects/config.json for controlling metadata source paths and package filtering in codegen/migration operationsmeta gen,meta verify, andmeta migratewith strict backward compatibility (projects without new config produce identical output)Risk Assessment
✅ Low: Both prior findings fixed correctly with regression tests; changes isolated to CHANGELOG documentation and migrate-scope refusal logic.
Testing
Executed 13 targeted test suites covering source resolution, scope filtering, both critical bug fixes, backward compatibility, and the governing invariant. All 79 tests passed with 179 assertions and 0 failures. Created comprehensive documentation artifacts demonstrating feature capabilities, pattern matching examples, and critical bug fix verification. No manual verification needed - automated tests provide complete coverage of user intent requirements.
Evidence: Test Summary Report
Evidence: Feature Demonstration Guide
Evidence: Test Execution Log
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
🔧 **Rebase** - 2 issues found → auto-fixed ✅
CLAUDE.md- merge conflict rebasing onto origin/mainCLAUDE.md~cd850b9ce (docs: metadata sources, scope, discovery, and the vendoring workflow)- merge conflict rebasing onto origin/main🔧 Fix applied.
✅ Re-checked - no issues remain.
🔧 **Review** - 2 issues found → auto-fixed ✅
CHANGELOG.md:138- The [Unreleased] bullet 'Discovery stops at ametaobjects/directory, not only at a config. A nested project holding its ownmetaobjects/and no.metaobjects/config.jsonkeeps reading its own metadata, as it always did, rather than adopting an ancestor's model andoutDir' describes the controller ruling that commit 39678a7 deliberately withdrew and reverted. The shipped code (sdk/src/discovery.ts) stops the walk only at.metaobjects/config.jsonand.git, and the branch's own tests pin the opposite behavior (sdk/test/discovery.test.ts:66 'a LOCAL metaobjects/ does not stop the walk — the ancestor config governs'; sdk/test/collection.test.ts:105 'a bare metaobjects/ is NOT a project boundary — the ancestor config governs'), as does docs/features/metadata-sources.md's Upgrading section. The intent requires the CHANGELOG to mirror the Upgrading section; instead it misinforms adopters on the exact load-bearing question of which metadata a nested config-less run resolves — an adopter believing this bullet would expect their subdirectory to keep its own model while it now silently adopts the ancestor's sources and outDir. The bullet was added in bdef4fc (before the revert) and never removed.server/typescript/packages/cli/src/lib/migrate-scope.ts:73- The empty-match refusal checksroot.objects(), which includesobject.valuenodes (the filter is by TYPE_OBJECT, not subType), so amigrate.scopewhose patterns match only non-persistable objects — e.g.acme::common::**where that package holds only shared value objects and abstracts — passesfqns.some(inMigrateScope)and dodges the refusal whilescopeExpectedSchemagoverns zero tables: the run compares nothing and reports 'no changes'. Mitigations keep this informational:outOfScopeNotestill prints naming every excluded object on both migrate and verify, and the structuraldeclaredSchemaspin (computed from the unscoped snapshot) prevents any destructive proposal, so this only weakens the second lock, not the invariant. A sturdier check would test against the built expected schema's provenance keys rather than all loaded objects.🔧 Fix: address stale changelog bullet and migrate scope provenance refusal
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
bun test packages/sdk/test/collection.test.ts- Collection resolution (13 tests)bun test packages/sdk/test/scope.test.ts- Scope pattern matching (12 tests)bun test packages/sdk/test/source-order.test.ts- Critical bug fix #1: source ordering (7 tests)bun test packages/sdk/test/scope-conformance.test.ts- Cross-port conformance corpus (11 tests)bun test packages/sdk/test/sources.test.ts- Source resolution (13 tests)bun test packages/sdk/test/no-hardcoded-metadata-dir.test.ts- Governing invariant enforcement (5 tests)bun test packages/cli/test/migrate-scope.test.ts- Critical bug fix #2: migrate.scope (5 tests)bun test packages/cli/test/integration/verify-db-scope.test.ts- DB scope verification (3 tests)bun test packages/migrate-ts/test/scope-snapshot-gate.test.ts- Snapshot scope gate (3 tests)bun test packages/codegen-ts/test/template-codegen/scope-walk.test.ts- Codegen scope walk (4 tests)bun test packages/cli/test/integration/gen-scope.test.ts- Codegen scope integration (2 tests)bun test packages/cli/test/integration/gen-sqlite.test.ts- Backward compatibility for codegen (4 tests)bun test packages/cli/test/integration/migrate-sqlite.test.ts- Backward compatibility for migrations (4 tests)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.