Skip to content

What a framework and its ecosystem library still hit in the engine: the #630 batch - #631

Merged
Sunrisepeak merged 29 commits into
mainfrom
630/engine-batch
Sep 13, 2026
Merged

What a framework and its ecosystem library still hit in the engine: the #630 batch#631
Sunrisepeak merged 29 commits into
mainfrom
630/engine-batch

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

Closes #630. One PR for the engine side of the ten items, as the design record .agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md lays them out (the record is in this PR).

What changes, by item

item change criterion
1, 2 one identity, two declarations: ResolvedRecord gains sourceRef and fromRoot; the root's declaration wins across kinds and references with a dependency/source-override warning naming both requesters and both references; a dependency's version requirement is checked against the root's checkout; two non-root requesters of different kinds are still refused, with the hint to declare the identity in the root; docs/05 en and zh gain the decision table e2e 661, six cases in both directions
3a mcpp pack stages the program and the declared files before the closure walk; a format whose closure cannot be walked on this host hands the tree over with closure = "not-walked" in the stage manifest; tar/dir still refuse e2e (in this PR after the pack commits land), unit tests for the stage manifest
3b a Mach-O reader for needed_names (thin and fat, LC_LOAD_DYLIB/LC_RPATH), the Mach-O row of is_system_lib, an @rpath resolver; bundling is not attempted unit tests with generated Mach-O fixtures
4 the C++ layer answers for its own headers: cxxFromGraph replaces cAbi.prebuilt() where the question was the payload's libc++; the iOS rows take llvm.libcxx and llvm.compiler-rt-builtins from the graph; without the package an Apple cross target takes the SDK's headers and import std is refused with the package named; the payload's missing iOS builtins archive is reported once as a degradation e2e 663 on Linux (a prebuilt C library under a graph libc++, both directions); ci-macos-ios builds and runs the simulator program over the packages and asserts the negative direction
5 min_api_level is a known [target.<triple>] key; a unit test's denominator is the parser's own body.find sites e2e 641 case 9 under --strict
6 the tool store's key holds the source: a git tool by its commit, a path tool by a stamp of its tree; upstreamKeys likewise e2e 665 (an edit and its reversal each reach the consumer; an unchanged tree is a store hit; e2e 187 unchanged)
7 an OS-only selector is a platform for mcpp emit xpkg; the residual warning says so unit tests for the predicate and the emitted descriptor
A9 an app whose artifact is a shared object takes several --target triples into one staged tree (lib/<abi>/) e2e on the Android rows
A8 declined again; the rule is stated in the record -

Ecosystem, already published

Measured

Root unit suite: 116 binaries, 0 failed. e2e on the integration binary: 641, 661, 663, 665, 187, 649, 330, 24, 32, 33 all pass. The macOS measurement is this PR's ci-macos-ios job.

…#630 item 5)

The unknown-key sweep's kKnownTargetScalars list drifted from the parser: it
read min_api_level (toml.cppm) but never listed it, so a correctly spelled
key was reported as unsupported and --strict turned that report into a build
failure. Adds the key to the list and to the message, extends 641 with the
manifest-only regression (a manifest that declares min_api_level on one row
must not trip --strict when building a different row), and adds a unit test
whose denominator is read from toml.cppm's own body.find(...) parse sites
rather than a second hand-written list, in both directions, so a sixth key
added without a matching list entry fails the test instead of shipping.
A [target.<selector>] tool declaration whose predicate names only an
operating system (cfg(linux), cfg(os = "linux"), the windows/macos/unix
equivalents, and cfg(unix) naming both linux and macosx) answers the exact
question a descriptor's per-platform block already asks, so emit_xpkg now
folds it into that block instead of only raising the publish/target-axis-tools
advisory. Any predicate mentioning an architecture, an environment, a layer,
or wrapped in any combinator (any/all/not, even one built entirely from OS
terms) keeps the warning, whose text now says so.

cfgpred::os_only_platforms is built on the existing cfg() Parser (two more
optional taps: seenKV and combinator) rather than a second reading of the
predicate text, per the "one grammar, two readers" rule this repository
already follows elsewhere. Unit tests cover both directions for the
classifier and for emit_xpkg's rendered output.
A second `git`/`path` declaration of an already-resolved dependency
identity carried no comparable reference: `ResolvedRecord` held no path
and no `gitRev`, so a `git`/`git` or `path`/`path` disagreement fell
through silently and the winner was whichever request the FIFO worklist
happened to dequeue first. A `path`/`git` KIND clash refused outright even
when the root itself was a party.

`ResolvedRecord` gains `sourceRef` (the declared git ref, the canonical
path, or the SemVer constraint) and `fromRoot`, so the resolve hit can
compare a second declaration's REFERENCE rather than only its kind, and
can tell whether the root is one of the two requesters. The root's
declaration now wins a kind clash or a same-kind reference conflict it is
a party to, reported through `mcpp::diag::warning`
("dependency/source-override") naming both requesters and both
references; a losing `version` requirement is checked against the
winning checkout's own `[package] version` and refused if violated, in
the same Holds/Violated shape `addrset::unify` already uses for a tool
pin. A kind clash between two non-root requesters keeps the existing
refusal, with one added hint sentence. The "root arrives after a
dependency" ordering is provably unreachable under FIFO worklist seeding
(root deps are seeded before the loop starts); rather than depend on that
silently, it is refused by name so the invariant cannot degrade into
today's accident of queue order.

docs/05 gains the decision table under a heading that finally answers
the "what happens when two of them disagree" its own preamble promises,
mirrored in docs/zh/05.

tests/e2e/661 builds a local git repository (three commits, then an
uncommitted edit) with no network, and covers all six rows of the
decision table plus the negative direction (the same reference declared
twice must stay silent, and `--strict` must stay green).
…std is not imported, and a reported degradation when it is
`pack::run` used to refuse a Mach-O program, and a non-PE artifact on a
Windows host, before any staging ran at all -- so a dispatched format that
names one program and never reads the dependency closure (an `.app`/`.msi`
bundler) had no tree to work from either, on a host where the built-in
closure walk cannot run.

The tree is now staged unconditionally -- the program, then the runtime
files `mcpp::deploy`/`[runtime] deploy` declared -- and the closure walk is
one step within it whose outcome is recorded rather than a precondition for
reaching that step. `pack::run` returns a `ClosureResult` (`walked` plus a
`reason` when not) instead of `void`; `closure_unavailable_outcome(Format)`
is the pure decision of what an unavailable closure means per format:
`--format tar`/`--format dir` still fail the command (the archive IS the
closure, unchanged), a dispatched format receives the tree regardless. The
stage manifest gains a `closure = walked | not-walked` header line, with
`reason = <why>` when not walked, so a provider can read it instead of
inferring a gap from an empty `lib/`.

The ELF product of `--format dir` is unaffected: the reorder only moves
where the Mach-O/Windows-host refusals are decided, not what the ELF path
does once it is reached.

Tests: unit coverage for `closure_unavailable_outcome` and for the stage
manifest writer/reader with both `ClosureStatus` values (the "not-walked"
case cannot be produced on Linux through `pack::run` itself -- it needs a
Mach-O program or a Windows host packing a non-PE artifact, covered
end-to-end by 266_pack_refuses_a_macho_program.sh on macOS CI); a new e2e,
662_pack_stages_declared_files_before_the_closure.sh, asserting the staged
tree's exact structure for `--format dir` and for a dispatched format's
action reading `${mcpp.stage_dir}`.

Docs: docs/30-build-mcpp.md and the zh mirror gain the `closure` manifest
field in the `${mcpp.stage_dir}` section.
`needed_names` returned "not implemented" for Mach-O; `elf_needed` and
`pe_needed` already read a dependency list without executing anything.
Mach-O gets the same treatment:

- `detail::macho_thin_needed` walks a thin object's load commands:
  `LC_LOAD_DYLIB`, `LC_LOAD_WEAK_DYLIB`, `LC_REEXPORT_DYLIB` and
  `LC_LOAD_UPWARD_DYLIB` contribute names, `LC_RPATH` contributes search
  entries, in load-command order. Both widths (`mach_header`/
  `mach_header_64`) and both endiannesses (the CIGAM magics) are handled.
- `detail::macho_needed` / the exported `macho_needed(path, arch)` add the
  FAT (universal) case: a slice is selected by `cputype` against the
  caller's `arch` (mcpp's canonical spelling), defaulting to the first
  slice when none is given -- which is what `needed_names` does, since it
  has no triple to pass. `FAT_MAGIC` and `FAT_MAGIC_64` are both handled;
  FAT headers are always big-endian on disk regardless of a slice's own
  endianness.
- `needed_names` completes for Mach-O by delegating to `macho_needed`, and
  `is_system_lib` gains a Mach-O row: `/usr/lib/` and `/System/Library/`
  are the operating system's, case-sensitively (`@rpath/...` is a search,
  not a root, so it is never system).
- `resolve_macho_names` is a pure resolver: `@executable_path`,
  `@loader_path` and `@rpath/<x>` (tried against every rpath entry, in
  order, with the same two substitutions) resolve to the first path that
  exists; a name that resolves nowhere is reported as `unresolved`, not
  dropped.

NOT wired into `pack::run`'s Mach-O closure step. Bundling a resolved
dylib beside the program and rewriting its `LC_RPATH` needs a load-command
editor -- a load command has no free space to grow a longer path into --
and the record (§4.2) designs that editor after resolution is measured on
a real macOS build, not before. `pack::run`'s Mach-O branch still reports
"not-walked" (see the sibling #630 §3 commit); this reader and resolver
are exposed for that measurement and for whoever wires them in next.

Tests: unit fixtures built in-process (a tiny Mach-O/fat-Mach-O byte
writer, no external tools, same shape the ELF/PE fixtures already use) --
a thin arm64 object with two `LC_LOAD_DYLIB`, one `LC_LOAD_WEAK_DYLIB` and
two `LC_RPATH`; a fat x86_64+arm64 object whose slices carry different
names, asserting the wrong slice is never read; a big-endian-magic thin
object; the wrong-arch request on a thin file (ignored, as documented);
`is_system_lib`'s Mach-O row; and the resolver's rpath-order and
unresolved-reporting behavior.

Docs: docs/10-pack-and-release.md and the zh mirror note that the closure
is read now, and narrow the "planned" gap to bundling + the `LC_RPATH`
rewrite.
The route mcpp pack takes is a function of the artifact's FORM, not of the
target's kind. A `kind = "app"` target whose resolved link form is a shared
object on every requested row (every Android row) now accepts more than one
`--target`, the way a library target already does: the other legs are built
first (`build_extra_android_legs`, the same leg-loop shape
`build_and_pack_library` uses) and staged as `lib/<abi>/lib<name>.so` beside
the primary leg's own `lib/<abi>/`, into one tree, behind one dispatch pass.
A `bin` target, or any row whose form is an executable, keeps today's
refusal for a second `--target`.

`toolchain::triple::android_abi` derives the ABI name from the triple's
architecture (aarch64 -> arm64-v8a, x86_64 -> x86_64, per the 2026-09-12
design record). `PackRoute::isApplication` and the pure predicate
`accepts_several_targets` let `cmd_pack` decide the route before building
anything. `Plan::extraSharedLegs`, empty for every pre-existing caller,
keeps a single-`--target` pack's staged tree byte-identical.

Tests: unit coverage for `android_abi` and `accepts_several_targets`
(8 cases), and e2e 664 exercising the staged tree for one and two
`--target` flags and both refusal directions on the sandbox's android-ndk
capability. docs/10 and its zh mirror gain the shape and layout.
@Sunrisepeak
Sunrisepeak marked this pull request as ready for review September 13, 2026 15:10
@Sunrisepeak
Sunrisepeak merged commit 6fb308b into main Sep 13, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants