From 4f4cff015bba819de811503c879a5e98987cefa0 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Thu, 10 Sep 2026 00:18:27 +0800 Subject: [PATCH 1/4] fix(runtime, build): the dlopen surface is walked, and one unwinder per process (2026.9.10.1) A SYCL project built cleanly and then terminated with exit code 134 and no exception text (#596). The trigger was in the ecosystem -- an adapter's farm was missing one driver library -- but the reasons it presented as a silent abort are two independent gaps in mcpp, and both are repaired here. THE SURFACE NOTHING WALKED. `resolve_runtime_closure` is seeded with the artifact and follows DT_NEEDED. A library a package publishes through `runtime.library_dirs` exists precisely because something will dlopen it, so no link edge names it and it is outside that closure BY CONSTRUCTION. Measured: a farm of twenty-five libraries, two of which could not load at all, while the build reported nothing -- because nothing had asked. `inspect_dlopen_surface` reads each such library's own DT_NEEDED and resolves it against the search path the artifact actually carries, separating three states: resolved, present as a dangling link (the machine has no driver), and absent (a packaging gap). Only the third is reported, and it is a warning: a dangling link is the documented shape of a host driver that is not installed, and failing there would turn a supported configuration into a failed build. `runtime.dlopen_surface` in resolution.json carries the findings and both denominators -- a surface that failed to build enumerates nothing, and "no findings" must not read like "nothing was examined". WHAT "THE SEARCH PATH THE ARTIFACT ACTUALLY CARRIES" TURNED OUT TO MEAN. Three corrections, all of them false positives, none visible until a project with a shared dependency was measured: * `$ORIGIN` leads every artifact's DT_RPATH and a shared dependency is deployed BESIDE the executable. `runtime_search_dirs` cannot carry that -- `$ORIGIN` is a property of each artifact, not of the plan -- so the artifacts' own directories are added in `check_dlopen_surface`. * A plan that produces no program has no surface to judge. The adapter package is `kind = "lib"`; reporting a consumer's surface against an archive's non-existent search path named a library the consumer resolves. * A SONAME is not a filename. mcpp links `bin/libopencl.so` whose SONAME is `libOpenCL.so.1`, and the alias appears later; `mcpp test` calls the check twice and only the second call saw it. The SONAMEs this build produces are read from the objects and passed in. TWO UNWINDERS IN ONE PROCESS. A lane whose device compiler is configured against libstdc++ puts libstdc++ on the link line while the artifact links libc++ statically. `hide_static_cxx_runtime` skipped executables on the premise that "ld exports only what a loaded object references, and mcpp passes no -rdynamic" -- a correct premise with a wrong conclusion, because a loaded libstdc++ DOES reference them. Measured: 89 exported symbols, 68 of them also defined by libstdc++ or libgcc_s. Ten of libgcc's eighteen `_Unwind_*` entry points came from the artifact and eight stayed in libgcc_s, including the accessors libstdc++'s personality routine calls. It read an LLVM libunwind context through libgcc's accessors, recovered a meaningless IP, found no landing pad, and `__cxa_call_terminate` ran past a handler three frames up; the verbose terminate handler's rethrow then terminated as well, which is why nothing was printed. Such a link now takes `--unwindlib=libgcc` and hides the static archives' symbols. libgcc_s is in the process either way -- libstdc++ needs it -- so this names a library rather than adding one, and the C++ runtime stays embedded. A link with no second runtime on it is byte-for-byte unchanged. Measured on one machine, same source, both situations: before exit 134, no output after `sycl: no usable device: ...` / `device unavailable`, exit 1 device `12 24 36 48` in both `-Wl,--exclude-libs` alone was written here as a prediction from the mechanism and then measured: exit 139 instead of 134. Hiding the exports makes libstdc++ bind to libgcc while the artifact's own libc++abi still calls its statically linked libunwind, so the mismatch reproduces in the other direction. The reason is recorded in the design record so the flag is not proposed again. The duplicate-symbol warning now states that consequence when the conflicting set includes the unwinder family, instead of describing it as one more copy that is never called. One documented claim was refuted along the way: the SYCL example states that a missing device image is the one failure its island cannot turn into a return code. Measured by compiling it for sm_90 and running on an sm_89 device -- it was not outside the catches; no catch worked. The example's comment and README are corrected. Design record: .agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md Ecosystem halves: openxlings/xim-pkgindex#796 #798, mcpplibs/mcpp-index#375 #377 #378 --- ...-09-09-dlopen-surface-and-two-unwinders.md | 639 ++++++++++++++++++ .agents/docs/2026-09-10-596-verify.sh | 174 +++++ .agents/docs/README.md | 4 +- CHANGELOG.md | 56 ++ docs/33-authoring-an-adapter.md | 38 ++ docs/42-heterogeneous-builds.md | 23 + docs/zh/33-authoring-an-adapter.md | 34 + docs/zh/42-heterogeneous-builds.md | 19 + examples/09-heterogeneous/sycl/README.md | 29 +- examples/09-heterogeneous/sycl/app/mcpp.toml | 8 +- .../sycl/app/src/kernels/saxpy.sycl | 27 +- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/distribution.cppm | 71 +- src/build/flags.cppm | 17 + src/build/ninja_backend.cppm | 40 ++ src/build/runtime_validation.cppm | 121 ++++ src/build/symbol_provision.cppm | 37 +- src/runtime/elf.cppm | 133 ++++ tests/unit/test_distribution.cpp | 69 ++ tests/unit/test_elf_runtime.cpp | 96 +++ 21 files changed, 1610 insertions(+), 29 deletions(-) create mode 100644 .agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md create mode 100755 .agents/docs/2026-09-10-596-verify.sh diff --git a/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md b/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md new file mode 100644 index 00000000..f57be4b1 --- /dev/null +++ b/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md @@ -0,0 +1,639 @@ +--- +subject: heterogeneous +status: active +--- + +# A dlopen surface no closure walks, and a process with two unwinders + +mcpp#596 reports that `examples/09-heterogeneous/sycl` builds cleanly and then +aborts with exit 134 on an NVIDIA machine, printing no exception text. The +reporter located the trigger exactly: `compat.sycl-runtime`'s farm carries +`libcuda.so.1` and not `libnvidia-ml.so.1`, so the SYCL runtime's CUDA adapter +cannot load. That is correct, and it is one of three mechanisms. + +This record separates them, because they are repaired in three different +repositories and only one of them is about NVML: + +* **A.** The libraries a package publishes through `runtime.library_dirs` exist + precisely because something will `dlopen` them. No closure mcpp walks reaches + them, so a farm that cannot satisfy its own members measures green. Two of + this farm's twenty-five members cannot. +* **B.** `compat.sycl-runtime`'s `install()` **enumerates** the payload's + library directory and **hand-writes** the driver's name. The hand-written + half is the half that is wrong, and the package's own criterion hand-writes + the same three names it checks. +* **C.** The artifact links LLVM libunwind statically and loads libgcc_s at run + time. Ten of libgcc's eighteen `_Unwind_*` entry points are interposed by the + executable and eight are not, so one throw is processed by two unwinders and + reaches `std::terminate` past a matching handler. **This is why the failure + prints nothing**, and the reporter set it aside as an unrelated build + warning. + +A is the reason mcpp did not catch it. B is the reason it exists. C is the +reason it presented as a silent abort rather than as one line of text. + +## 1. What was measured + +Host: Linux 6.8, x86_64, RTX 4080 (sm_89), NVIDIA driver 550.144.03. mcpp +`2026.9.8.1` (released binary, its own provisioned registry), `mcpp:plugins` +0.5.2, `xim:dpcpp` 7.1.0, `compat.sycl-runtime` 2026.09.07. The project is +`examples/09-heterogeneous/sycl`, unmodified. The reporter's host is a +different GPU (4070 Ti SUPER) on a different driver (610.57.04) with mcpp +`2026.9.9.1`; the readings below agree with theirs except where noted in §2.3. + +| # | Input | Reading | +|---|---|---| +| 1 | `mcpp build` | green in 2.70 s, no runtime-closure diagnostic | +| 2 | `./bin/sycl-saxpy` | `terminate called ...` / `terminate called recursively`, exit **134** | +| 3 | `SYCL_UR_TRACE=1` | cuda adapter fails on `libnvidia-ml.so.1`; opencl adapter fails on `libOpenCL.so.1`; both level_zero adapters load | +| 4 | `ln -s /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1 /` then run | `12 24 36 48` / `device: NVIDIA GeForce RTX 4080` | +| 5 | remove that link, run again | back to row 2 | +| 6 | `/ld.so --help` | search path is the literal `/nonexistent/xlings-use-rpath-not-default-search/lib` | +| 7 | `nm -D` on the artifact vs. libgcc_s | 10 of 18 `_Unwind_*` names defined by the executable | + +Rows 4 and 5 are the pair that matters: one symbolic link, both directions, +same binary, same machine. Row 6 is what makes row 3 fatal rather than +cosmetic — the private loader consults no host directory on any distribution, +so "absent from the farm" is "absent from the machine". + +The same objects were then relinked by hand, four ways, and each variant was +run in both situations — the unrepaired farm (no device reachable) and the repaired +one (the 4080 reachable). This is the evidence §6's R6 rests on: + +| Variant | Unwinders | Exported / overlapping | When it fails | With a device | +|---|---|---|---|---| +| V0 — as mcpp links today | 2 | 89 / 68 | `terminate`, **exit 134**, no text | `12 24 36 48`, exit 0 | +| V1 — `--unwindlib=libgcc`, no `libunwind.a` | 1 | 58 / 58 | **`sycl: no usable device: No device of requested type available.` / `device unavailable`, exit 1** | `12 24 36 48`, exit 0 | +| V2 — V1 + `-Wl,--exclude-libs,ALL` | 1 | **0 / 0** | same as V1, exit 1 | `12 24 36 48`, exit 0 | +| V3 — `--exclude-libs,ALL` alone | 2 | 0 / 0 | **SIGSEGV, exit 139** | `12 24 36 48`, exit 0 | + +V3 is the variant that looks right. It removes every duplicate symbol, its +run with a device is perfect, and it turns a silent abort into a silent +segfault. The failing run is the only place the difference is visible, which is the same +property that let V0 ship. + +Rows 4, 5 and the variant runs were performed against a scratchpad registry and +reverted; the working tree carries no change from this investigation. + +## 2. Mechanism A — the declared dlopen surface is outside every closure + +### 2.1 The walk + +`resolve_runtime_closure` (`src/runtime/elf.cppm:920`) seeds its queue with the +artifact and one thing only (`elf.cppm:943`), then follows `DT_NEEDED`. An +unresolvable SONAME is recorded (`elf.cppm:979`) and, under a hermetic binding, +becomes `Unresolvable` with the message that names the private loader +(`elf.cppm:1229`). + +`libur_adapter_cuda.so.0` is never a `DT_NEEDED` of anything in that closure. +`libsycl.so.9` loads `libur_loader.so.0`, which `dlopen`s each adapter — the +trace in row 3 shows it trying the bare SONAME first and then the absolute path +inside the farm. Nothing in the artifact's link-time graph names it, so it is +outside the walk **by construction**, not by oversight. + +Measured, with the artifact's real `DT_RPATH` as the search path: + +``` +libur_adapter_cuda.so.0 FAIL libnvidia-ml.so.1 +libur_adapter_opencl.so.0 FAIL libOpenCL.so.1 +libur_adapter_level_zero.so.0 ok +libur_loader.so.0 ok +libsycl.so.9 ok +``` + +### 2.2 mcpp already holds every input this check needs + +`runtime_search_dirs` (`src/build/runtime_validation.cppm:449`) assembles the +exact directory list the artifact will use, and it is already passed to the +closure walk at `runtime_validation.cppm:676`. `plan.depRuntimeLibraryDirs` +(`src/build/plan.cppm:265`, filled at `plan.cppm:1124`) is the set of +dependency `runtime.library_dirs` — that is, the set of directories a package +published *because* its contents are reached by `dlopen` rather than by a +header or a link line. + +So the missing check is not a missing capability. It is a missing edge: the one +surface mcpp itself put on the search path is the one surface it does not walk. + +### 2.3 The second gap, and one disagreement between hosts + +`libur_adapter_opencl.so.0` needs `libOpenCL.so.1`, which the farm does not +carry either. On this host that adapter fails; on the reporter's host the trace +shows it **loading**, and the Intel CPU device it enumerated is what their +default selector then chose. Some path on their machine supplies +`libOpenCL.so.1` and it is not visible from here. The gap is real on both — the +farm carries no ICD loader — but its consequence is host-dependent, and that +disagreement is unexplained. See §7. + +This also refines the reporter's causal chain in one place. The abort does not +depend on a wrong device being selected. On this host **no device existed at +all** and the first thrown exception is `No device of requested type +available.`; the observable outcome is identical, down to the exit code. "The +default selector picked the CPU" is a companion symptom of the same missing +adapter, not a link in the chain. + +### 2.4 Why the package's own criterion is green + +`mcpp-index`'s `tests/examples/sycl-runtime/tests/farm.cpp` exists to assert +that this farm works. It `dlopen`s three sonames: `libsycl.so.9`, +`libur_loader.so.0`, `libumf.so.1`. The farm has twenty-five entries. The two +that are broken are not among the three. + +The reason its comment gives for not testing the adapters is that "a machine +with no GPU is a legitimate configuration and is what every runner in this +repository is". That reason does not hold: **whether an adapter can be +`dlopen`ed is a packaging property, not a device property.** This host has no +Level Zero device and both Level Zero adapters load. The two that fail report +`cannot open shared object file`, which is a statement about the farm, not +about the hardware. + +There is a real obstacle behind the wrong reason, and it must be handled or the +repaired test will be red on every runner and will be reverted: on a machine +with no NVIDIA driver the farm's `libcuda.so.1` is a **deliberately dangling** +symlink — `xim:libcuda-host-link` creates it pointing at the canonical path so +that installing a driver later self-heals every consumer. A naive "dlopen +everything" is red there. §5 states the three-state rule that both this test and +the mcpp-side check need. + +## 3. Mechanism B — a hand-written name where the same function enumerates + +`compat.sycl-runtime`'s `install()` builds the farm in two halves: + +* the payload's libraries are **enumerated** — `ls -1 /lib`, link + every versioned SONAME; +* the driver is **hand-written** — one `ls` for + `xim-x-libcuda-host-link/*/lib/libcuda.so.1`, one link, one name. + +The recipe's own header explains, at length and correctly, why the driver has +to be in this farm: once the payload's libraries acquired `RUNPATH = $ORIGIN`, +a non-empty RUNPATH switched off the inherited `DT_RPATH` for their +dependencies (mcpp#460), so the adapter can no longer reach a driver two farms +away. Every word of that reasoning applies to `libnvidia-ml.so.1` unchanged. +The defect is not in the reasoning; it is that the conclusion was written as a +name instead of as a set. + +Two ecosystem constraints bear on the repair, and they rule out one of the two +options the issue proposes. + +**GPU-related index packages are not permitted to probe the host.** The +sentinels `xim:libcuda-host-link` and `xim:nvidia-gl-host-link` are the single +source of truth for where the driver is; `hostlib.lua` records the history — +four independent probes, three of them wrong, one returning a 32-bit +`libcuda.so.1` on a biarch host and failing three layers away (mcpp#352). The +issue's option 2 — copy `farm_libc_stubs` and change the library name — looks +like precedent but is not: that helper searches the **xpkg store**, and +`libnvidia-ml.so.1` is not in any store. Copying it would produce a host probe, +which is the forbidden shape, and would bypass the ELF-class check and the +self-healing dangling-link semantics the sentinel already implements. + +**A version key selects the anchor URL and does not freeze behaviour.** There is +one `install()` in that recipe and it never reads `pkginfo.version()`, so +installing `2026.09.06` today builds the farm `2026.09.07` builds. The +consequence for this repair is the opposite of what it sounds like: existing +pins pick the fix up automatically, but a machine that **already has** the +directory does not reinstall, so a new version key is still required to reach +installed hosts. + +## 4. Mechanism C — two unwinders in one process + +This is the part the issue does not contain, and it is the reason the failure is +as hard to diagnose as it is. + +The island declares a synchronous failure path +(`examples/09-heterogeneous/sycl/app/src/kernels/saxpy.sycl:114`): + +```cpp +} catch (const sycl::exception& e) { + std::fprintf(stderr, "sycl: no usable device: %s\n", e.what()); + rc = -1; +} +``` + +On this host that path should run — there is no usable device — and `main` +should print `device unavailable` and return 1. It aborts instead. The throw is +on the main thread, three frames below the handler: + +``` +#0 __cxa_throw (tinfo = typeinfo for sycl::_V1::exception) libstdc++ +#1 sycl::_V1::detail::select_device … libsycl.so.9 +#3 sycl::_V1::queue::queue<…> +#4 saxpy_device the frame whose caller has the catch +#6 main +``` + +At `abort`, the reason is explicit: + +``` +#3 __cxa_call_terminate +#4 __gxx_personality_v0 (actions=6) libstdc++ +#5 unwind_phase2 () 0x555555618… — the executable's LLVM libunwind +#6 _Unwind_RaiseException () the executable's +``` + +and one frame deeper, during phase 1: + +``` +#0 _Unwind_GetIPInfo (context=0x7fffffffc930) libgcc/unwind-dw2.c:360 libgcc_s +#1 __gxx_personality_v0 (actions=1, …) eh_personality.cc:457 libstdc++ +#2 _Unwind_RaiseException () 0x5555556181f6 LLVM libunwind +``` + +libstdc++'s personality routine reads an `_Unwind_Context` built by **LLVM +libunwind** through **libgcc's** accessor. The two structures are unrelated. The +IP it recovers is meaningless, the LSDA lookup at that IP finds no landing pad, +and phase 2 ends in `__cxa_call_terminate`. `__verbose_terminate_handler` then +rethrows in order to print the exception's type — through libgcc's +`_Unwind_Resume_or_Rethrow`, which is not interposed — and terminates again, +which is the `terminate called recursively` line and the reason no message is +printed. + +The split is measurable and is an artifact of static-archive granularity: + +``` +libgcc_s _Unwind_* entry points 18 +defined by the executable (libunwind.a) 10 +still resolved in libgcc_s 8 incl. _Unwind_GetIPInfo, + _Unwind_GetCFA, _Unwind_Resume_or_Rethrow +``` + +The link line is `-nostdlib++` plus `libc++.a libc++abi.a libunwind.a` named +explicitly, under `-stdlib=libc++ -rtlib=compiler-rt --unwindlib=libunwind`; the +artifact's `DT_NEEDED` is `libsycl.so.9`, `libstdc++.so.6`, `libm.so.6`, +`libc.so.6`, and libgcc_s arrives underneath libstdc++. The linker pulls from +`libunwind.a` only the members that something references and exports them +because a loaded shared library has undefined references to them; the other +eight names were never referenced by libc++abi and so were never pulled in. +Nothing chose ten; ten is what the archive resolution happened to need. + +### 4.1 The other fifty-eight + +The warning says 68, and the unwinder family is only ten of them. The rest are +libc++abi's and libc++'s copies of the `std::` exception root — `vtable`, +`typeinfo`, `typeinfo name`, the destructors and `what()` for `std::exception`, +`bad_alloc`, `bad_cast`, `bad_typeid`, `bad_exception`, `bad_array_new_length` +and `type_info` — plus, for `std::logic_error` and `std::runtime_error`, their +**constructors and `operator=`**. + +Those last two are the same partial-interposition shape as the unwinder, moved +from control flow to object layout. `std::runtime_error`'s constructors are +exported and its **destructor is not**. libstdc++-compiled code that constructs +one therefore builds a libc++ object — whose payload is a +`__libcpp_refstring` — and destroys it through libstdc++'s destructor, which +expects a `__cow_string`. Nothing in this example exercises that path, so it is +a hazard rather than a measured failure; it is recorded because the repair that +fixes the unwinder does not by itself remove it, and V2 does. + +Three consequences worth stating separately: + +1. **The condition is already detected.** mcpp's duplicate-symbol warning names + these symbols and names `libgcc_s.so.1` as the other provider. It classifies + the impact as "the library's own copy is never called". For the unwinder + family the impact is that exception handling does not work. +2. **The seam discipline cannot fix it.** The island rule — device code is + reached only through `extern "C"` — is about symbols crossing a boundary the + author writes. The unwinder is reached through the process-global symbol + namespace by code neither side wrote. +3. **It only shows on the error path.** When nothing throws, this artifact runs + correctly and prints `12 24 36 48`. That is why it shipped. + +The island's comment (`saxpy.sycl:22-31`) says the one failure it cannot turn +into a return code is a missing device image, "which is why the manifest names +the device — and why `mcpp.rules.sycl` warns at build time when it does not". +That mitigation was written against one cause. This issue is the second: the +manifest named the device, the image was compiled for that device, and the back +end that consumes it never loaded. The mitigation is conditioned on the +manifest, and the failure is conditioned on the runtime. + +## 5. The three-state rule + +Both repairs that walk a farm need the same distinction, and it is stated once +here so that each can cite it and neither can be folded into the other: + +| State of a farm member's `DT_NEEDED` SONAME | Meaning | Report | +|---|---|---| +| resolves on the artifact's search path | nothing to say | silent | +| no entry anywhere on that path | **packaging gap** | name the member, the SONAME, and the package that published the directory | +| an entry exists in the farm but is a dangling symlink | **machine gap** — the sentinel's self-healing shape, no driver installed | distinct wording, never an error | + +A `DT_NEEDED` that is an absolute path (`nvidia-gl-host-link` deliberately +patches `/lib/x86_64-linux-gnu/libGLX_nvidia.so.0` into the GL farm) is resolved +as a path, not searched. A survey that searched it reported the GL and Vulkan +farms as broken; they are not, and that survey is not evidence about them. + +## 6. Repairs + +Seven, in three repositories. Each carries its own criterion, because a +requirement that shares another repair's criterion disappears when that repair +ships. + +### R1 — `xim-pkgindex`: the sentinel names a set + +`pkgs/l/libcuda-host-link.lua` probes and links one name. Make the name a list +— `libcuda.so.1`, `libnvidia-ml.so.1` — through the same `hostlib.path_of`, the +same ELF-class check, and the same canonical-path fallback for the +not-yet-installed case, so both links self-heal identically. + +*Criterion.* On a host with a driver, both files exist under the sentinel's +`lib/` and both resolve. On a host without one, both exist and both dangle, and +the package still installs successfully. Denominator: the recipe's name list, +asserted by count, so that a list that silently became empty is visible. + +### R2 — `mcpp-index`: `compat.sycl-runtime` enumerates the sentinel too + +Replace the hand-written `libcuda.so.1` lookup in `install()` with an +enumeration of the sentinel's `lib/` directory, filtered by the same +versioned-SONAME rule the payload half already uses (an unversioned name would +reach the linker, which the header explains). Publish a new version key so hosts +that already hold `2026.09.07` reinstall. `compat.cuda-runtime` carries the same +hand-written shape and should be converted with it. + +*Criterion.* Two checks, and the second is the one that fails today: +1. after install, `/libnvidia-ml.so.1` exists; +2. the private loader resolves the cuda adapter's whole `DT_NEEDED` against the + artifact's real search path — `ld.so --library-path "$RPATH" --list + /libur_adapter_cuda.so.0` prints no `cannot open shared object file`. + This must be run with the fix removed as well; it is red today and must be + red again if R2 is reverted. + +### R3 — `mcpp-index`: the farm's criterion enumerates its own members + +`tests/examples/sycl-runtime/tests/farm.cpp` checks three hand-written names. +Change it to enumerate the farm directory and apply §5 per member: dangling +entries are skipped with a printed note, missing SONAMEs fail. Print the number +of members examined. + +*Criterion.* The test reports a member count greater than zero (a farm that +failed to build must not read as a pass), fails on today's `2026.09.07` farm +naming `libur_adapter_cuda.so.0` and `libnvidia-ml.so.1`, and passes after R1 +and R2. It must also pass unchanged on a GPU-less runner, which is the check +that decides whether the three-state rule was implemented or merely written +down. + +*`dlopen` OF EACH MEMBER IS NOT ENOUGH, and this was measured rather than +foreseen.* The first implementation enumerated the farm and `dlopen`ed every +member, which is the obvious reading of "enumerate the population". It passed on +a farm with NVML removed, because `dlopen` measures the PROCESS: another farm on +the same search path supplied the missing soname (see R7). A package's test has +to be able to fail on that package alone, so the test reads each member's +DT_NEEDED itself and resolves it against the farm plus the three sonames the +artifact has already loaded. The three states are then decided from the farm's +own contents rather than from whatever else the run happened to have. + +*Measured, all three situations:* + +| situation | reading | +|---|---| +| repaired farm, driver present | PASSED, members 26, walked 26 | +| NVML removed from the farm | FAILED, names the member and the soname | +| driver links made dangling | PASSED, members 26, walked 24, two noted | + +### R4 — `mcpp`: the closure check reaches the declared dlopen surface + +Extend the artifact verdict: for each directory in `plan.depRuntimeLibraryDirs`, +read each ELF member and resolve its `DT_NEEDED` against the same `searchDirs` +already computed at `runtime_validation.cppm:676`, applying §5. This needs no +knowledge of SYCL, CUDA or Unified Runtime, which is what keeps it on the right +side of `test_runtime_contract`'s prohibition on branching in mcpp's source on a +provider's vocabulary. Cost is a few dozen ELF headers. + +Severity is **advisory, not fatal**. A farm legitimately holds host-driver links +that dangle, and the check must not turn a CPU-only machine's correct +configuration into a failed build. + +Publish the count of members examined into `resolution.json` alongside the +findings. A check whose "nothing found" and whose "nothing looked" read the same +is the failure mode this repository has named more than once. + +*Criterion.* A unit test in `tests/unit/test_elf_runtime.cpp` over a synthetic +directory holding three members — one whose `DT_NEEDED` resolves, one naming an +absent SONAME, one that is a dangling link — asserting three distinct outcomes +and the member count. Field assertions on the record, not substring matches on +the message. A second check builds the SYCL example against an unrepaired farm and +asserts the warning names `libnvidia-ml.so.1`; it does not need a GPU, which is +the point. + +*Ordering.* R4 lands **after** R1 and R2 reach the index. Landed first, it warns +correctly and loudly about a released package that no user can repair, and the +first thing it reports is our own fixture. + +*The check is configuration-sensitive by construction, which is the point.* Run +against this host's long-lived `~/.mcpp`, it reported the OpenCL gap and NOT the +NVML one -- because that registry's SubOS library view carries +`libnvidia-ml.so.1` through `xim:nvidia-gl-host-link`, so on that configuration +the adapter genuinely resolves. The reporter's machine had no such view. A check +that answered the same on both would be answering from a table rather than from +the search path, which is the failure it exists to remove. + +### R5 — `mcpp`: the duplicate-symbol warning states the real consequence + +When the duplicated set intersects the `_Unwind_*` family, the existing warning +should say that exception handling across the boundary will not work, rather +than that the library's own copy will not be called. Text only; no behaviour +change. + +*Criterion.* The SYCL example's build output contains the unwinder-specific +sentence, and a build whose duplicates are ordinary symbols does not. Two checks, +because a message that always appears carries no information. + +### R6 — `mcpp`: one unwinder per process, and no exported seam + +The invariant is that a process has exactly one unwinder. This artifact has two, +split by which archive members were referenced. + +The repair is **both halves of V2**, and the §1 matrix says why neither alone is +the answer: + +* `--unwindlib=libgcc`, dropping `libunwind.a` from the unit's link line, for an + artifact whose link line already names libstdc++. libc++abi and libstdc++ then + call libgcc's entry points on libgcc's contexts. Measured: the island's + handler runs, the program prints its diagnosis and exits 1. This is the half + that fixes the reported silence. +* `-Wl,--exclude-libs,ALL`, which drops the exported set from 58 to 0 and + removes §4.1's constructor-without-destructor hazard along with it. + +**`-Wl,--exclude-libs` alone is not a repair, it is a regression** — this was +written here as a prediction from the mechanism and then measured as V3: exit +139 instead of 134. Hiding the executable's `_Unwind_*` makes libstdc++ bind to +libgcc, but the executable's own libc++abi still calls its statically linked +libunwind internally, so an exception raised there and unwound through a +libstdc++-compiled frame reproduces the mismatch in the opposite direction. The +flag looks exactly right, its run with a device is clean, and it makes the failure +harder to read. Nobody should have to rediscover that. + +*Predicate.* The change applies to an artifact whose link line names libstdc++. +It is decidable where the link line is assembled, and it is narrow: an artifact +that does not load libstdc++ keeps today's hermetic `libunwind.a` and gains no +`DT_NEEDED` on `libgcc_s.so.1`. V1 and V2 both acquire that entry; it is +satisfied from the payload farm already on the search path, which is why both +variants run. + +*Criterion.* Four cells, all measured on this host and all four required: + +| | when it fails | with a device | +|---|---|---| +| before | exit 134, no text | `12 24 36 48` | +| after | `sycl: no usable device: …`, exit 1 | `12 24 36 48` | + +plus two structural assertions: mcpp's own duplicate-symbol warning does not +appear for this example after the change (it is the cheapest denominator +available — it counts the seam), and an artifact with no libstdc++ on its link +line is byte-for-byte unchanged. + +*Implemented as V4, which is V2 with the archives named.* `--exclude-libs,ALL` +was the measured variant; the shipped one lists `libc++.a` and `libc++abi.a`, +matching the spelling `hide_static_cxx_runtime` already used for shared +libraries, so a user's own static library linked into an artifact keeps its +exports. Measured to be equivalent for this purpose: 0 exported symbols, 0 +overlapping, exit 1 with the diagnosis when it fails, `12 24 36 48` with a +device. + +*The predicate lives where the mechanism table already is.* `MechanismInput` +gains `foreignCxxRuntime`, set in `flags.cppm` from `bc.ldflags` containing +`stdc++` while the toolchain's library is libc++; `distribution.cppm` reads it +in the libc++ ELF branch. The same field widens `hide_static_cxx_runtime` from +shared libraries to executables, correcting the comment quoted in §4. + +*One documented claim was refuted along the way.* The SYCL example states that a +missing device image is the one failure its island cannot turn into a return +code, "because it throws from inside the scheduler in neither of those two +paths". Measured by compiling the same example for `sm_90` and running it on an +sm_89 device: with two unwinders, `terminate called after throwing an instance +of 'ur_result_t'`, exit 134; with one, the island's own handler prints the +build log and `main` prints `device unavailable`, exit 1. It was not outside the +catches; no catch worked. The example's comment and README are corrected. + +*Status.* Measured and implemented. What is **not** measured is the scope +question in §7: `--exclude-libs` on a shared-library artifact is unchanged +behaviour, but an executable that deliberately re-exports an interface from a +static archive would now hide it -- no such artifact exists in this tree, and +the flag is emitted only for the libc++-plus-libstdc++ combination. + +### R7 — decide `libOpenCL.so.1` + +Either declare the install-time edge to `compat.opencl` and farm the ICD loader, +or record that the SYCL lane deliberately offers no OpenCL back end. The present +state — neither carried nor stated — is what let the reporter's machine select a +CPU device silently. + +*Criterion.* R2's second check applied to `libur_adapter_opencl.so.0`, plus the +one that made the reversal safe: with the edge declared, removing NVML from the +farm must still fail R3. + +*Decided: SERVED, after one round of deciding the opposite.* The history is +worth keeping, because the wrong decision was made from a real measurement. + +The edge was written -- `deps = { ["compat.opencl"] = "2026.05.29" }` -- and the +farm test then passed with every member loading. It also passed with +`libnvidia-ml.so.1` REMOVED from the farm. `compat:opencl` depends on +`compat:opencl-runtime`, whose farm mirrors the host's NVIDIA OpenCL family and +therefore carries NVML: the new dependency was satisfying the need R2 exists to +satisfy, and R2's criterion could no longer fail. On that reading the edge was +withdrawn and the adapter was recorded as unserved. + +**That reason expired the moment R3 was hardened.** The masking was a property +of a criterion that measured the PROCESS, and R3 was rewritten -- because of +this very measurement -- to read each member's DT_NEEDED against the farm +alone. Once nothing on the search path can answer for the farm, the only +objection left was the size of the surface, and "the payload ships an adapter +that can never load" is not something a runtime adapter should leave standing. + +So the edge is declared. Measured with it in place: the farm test passes, and +with NVML removed it still FAILS naming `libur_adapter_cuda.so.0` -- the +criterion is no longer maskable, which is what made the decision reversible. + +`libOpenCL.so.1` stays named in the test, with its meaning changed: not "this +package does not serve it" but "a declared dependency of this package provides +it". Everything not on that short list must be in the farm, and the list is +what keeps the self-sufficiency assertion exact without reopening the process +to answer for it. + +*And it found three defects in R4*, all of them false positives, and all of +them invisible until a project with a shared dependency was measured: + +1. **`$ORIGIN` is not in `runtime_search_dirs`.** Every artifact carries it + first in its DT_RPATH and a shared dependency is deployed BESIDE the + executable, so the library resolves at run time and read as missing here. + `runtime_search_dirs` cannot carry it: `$ORIGIN` is a property of each + artifact, not of the plan. The artifacts' own directories are added in + `check_dlopen_surface`. +2. **A plan that produces no program has no surface to judge.** The adapter + package is `kind = "lib"`, and reporting a consumer's surface against an + archive's non-existent search path named a library the consumer resolves. +3. **A SONAME is not a filename.** mcpp links `bin/libopencl.so` whose SONAME + is `libOpenCL.so.1`, and the alias under the SONAME appears later; `mcpp + test` calls the check twice and only the second call saw it. The SONAMEs + this build produces are read from the objects and passed in. + +After all three, the same project reports the same four findings on both calls, +and every one of them is real -- they are the `compat:vulkan-runtime` class +already filed as mcpp-index#376, reached this time through +`compat:opencl-runtime`. + +## 7. Open + +* **The OpenCL disagreement (§2.3).** The adapter loads on the reporter's host + and not here, from the same pinned payload. R7 now serves the adapter through + `compat:opencl`, so the practical consequence is gone; what is still + unexplained is why their machine supplied `libOpenCL.so.1` without it. +* **What R6 does to an executable that is itself a plugin host.** Hiding + `libc++.a` and `libc++abi.a` means a library `dlopen`ed later cannot resolve + the C++ standard library from the executable. That is the intended direction + — it is how the second runtime stops leaking — but a libc++-built plugin + loaded into such a process would now find no provider at all. The + configuration is narrow (it requires libstdc++ already on the line, which is + the broken state this repairs) and no artifact in this tree is one, so it is + recorded rather than handled. +* **Whether other farms have packaging gaps — answered, by R4 itself.** The + earlier survey here used a deliberately partial search path and its findings + were about the survey. R4 run per project answers it properly, and the first + thing it caught is this ecosystem's own `compat:vulkan-runtime`: 4 of 55 + libraries on a host with an NVIDIA driver. + + | library | needs | on this host | + |---|---|---| + | `libnvidia-encode.so.1` | `libnvcuvid.so.1` | present in `/usr/lib`, absent from the farm | + | `libnvidia-opticalflow.so.1` | `libnvcuvid.so.1` | same | + | `libnvidia-pkcs11-openssl3.so.550.144.03` | `libcrypto.so.3` | same | + | `libnvidia-pkcs11.so.550.144.03` | `libcrypto.so.1.1` | same | + + All four are real and all four are the same shape as mcpp#596 -- a farm that + mirrors a driver family and stops one library short. They are NOT repaired + here: a check whose first catch is its author's own package should report it, + not quietly absorb it, and the repair belongs to `compat:vulkan-runtime` with + its own criterion, filed as mcpplibs/mcpp-index#376. On a runner with no NVIDIA driver the farm is nearly empty + and the check is silent, so this does not appear in CI. + +## 8. Order + +R1 → R2 (R2 reads what R1 publishes) → R3 (asserts what R2 produced) → R4 +(would otherwise report a released package no user can repair). R5 and R6 are +independent of that chain and of each other; R6's second half is gated on the +artifact-kind question in §7, its first half is not. R7 landed with R2. + +R5 and R6 are worth stating as one sentence, because they are the same finding +seen from two sides: R6 removes the seam, and R5 is what should have been said +about it while it was still there. Neither is a consequence of #596 — the seam +predates it, the example's README announces it, and the example runs correctly +across it every time nothing throws. #596 is simply the first time something +threw. + +## 9. What this touches, across the three repositories + +The review that asks the other question: not "is each repair right" but "what +else moves when they land". + +| Change | Who reads it | What happens to them | +|---|---|---| +| `xim:libcuda-host-link` 0.0.2 | `ollama` pins 0.0.1 | unchanged; 0.0.1 is kept and still resolves | +| | `compat.cuda-runtime` pins 0.0.1, frozen | untouched, deliberately: its header says it receives no new versions | +| | `compat.cuda-driver`, `compat.sycl-runtime` | move to 0.0.2, which is the only place that decides which sonames they get | +| `compat.*` 2026.09.10 | consumers pinning 2026.09.05/07 | still resolve; a machine that already holds the directory keeps the old farm until something asks for the new key | +| | `examples/09-heterogeneous/sycl` | pin moved, in the mcpp change | +| R4 (the surface walk) | every Linux build with a dependency `runtime.library_dirs` | a warning where there is a real gap. Measured: 0 for a project with no such dependency, 1 for the SYCL example (the declared-unserved OpenCL adapter), 4 for the Vulkan example on a host with an NVIDIA driver (real, filed as mcpp-index#376) | +| | a non-hermetic binding, or `allow_host_libs` | silent, for the reason the artifact verdict is | +| R6 (one unwinder) | a libc++ link line naming libstdc++ | `--unwindlib=libgcc` and hidden archives | +| | every other link | byte-for-byte unchanged, asserted in `test_distribution.cpp` | + +The one regression this could cause is in §7's last item: an executable that +deliberately re-exports the C++ standard library to a plugin it `dlopen`s. It +requires libstdc++ already on the line, which is the broken state R6 repairs, +and no artifact in this ecosystem is one. diff --git a/.agents/docs/2026-09-10-596-verify.sh b/.agents/docs/2026-09-10-596-verify.sh new file mode 100755 index 00000000..6b290b61 --- /dev/null +++ b/.agents/docs/2026-09-10-596-verify.sh @@ -0,0 +1,174 @@ +#!/usr/bin/env bash +# Ecosystem verification for mcpp#596: the driver farm mirrors the sentinel, +# and mcpp reports a dlopen surface it cannot satisfy. Against a PUBLISHED +# mcpp, a PUBLISHED xim:libcuda-host-link and a PUBLISHED compat:sycl-runtime. +# +# # The sandbox has an EMPTY $HOME and a fresh /tmp, so this file is not +# # visible from inside it. Pass the script itself in: +# B64=$(base64 -w0 ) +# xlings subos use verify-596 --sandbox --cmd \ +# "echo $B64 | base64 -d > /tmp/v.sh && MCPP_VERIFY_VERSION=2026.9.10.1 bash /tmp/v.sh" +# +# mcpp is addressed by its STORE path, which is the one thing the sandbox does +# share: the xlings data directory. A bare `mcpp` is not on PATH in there. +# +# WHAT A SANDBOX CAN AND CANNOT DECIDE HERE. +# +# It decides everything about the PACKAGING, which is where the defect was: how +# many driver sonames the sentinel publishes, and whether the farm carries all +# of them. Those are properties of what gets installed, and a machine that has +# built this before answers them from a directory that already existed. +# +# It cannot decide anything about a DEVICE. The sandbox's /dev has fourteen +# entries and no NVIDIA node, so `libcuda.so.1` resolves to a dangling link +# there exactly as it does on any machine without a driver -- which is a +# supported configuration and is asserted as such below, not worked around. The +# device side is measured on the host and recorded in the design record. +# +# EVERY CRITERION NAMES THE OBJECT IT SELECTED, and every section that did not +# run is listed again in the summary. "0 assertions failed" printed by a script +# that skipped three sections is the failure mode this shape exists to prevent. +set -u + +VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" +STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" +SENTINEL_VER="${MCPP_VERIFY_SENTINEL:-0.0.2}" +COMPAT_VER="${MCPP_VERIFY_COMPAT:-2026.09.10}" + +fails=0 +skipped="" +fail() { printf 'ASSERT-FAIL: %s\n' "$1"; fails=$((fails + 1)); } +ok() { printf 'ok: %s\n' "$1"; } +section() { printf '\n== %s ==\n' "$1"; } +skip() { printf 'NOT RUN: %s\n' "$1"; skipped="$skipped + - $1"; } + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +section "A. the published mcpp answers for itself" +if [ ! -x "$STORE" ]; then + fail "no mcpp at $STORE" + printf '\nfails=%d (nothing else can run)\n' "$fails" + exit 1 +fi +got=$("$STORE" --version 2>&1 | head -1) +case "$got" in + *"$VER"*) ok "mcpp --version says $got" ;; + *) fail "mcpp --version says '$got', expected $VER" ;; +esac + +# --------------------------------------------------------------------------- +section "B. the sentinel publishes a SET, and the size is the assertion" +# +# The count is the criterion rather than the presence of one name: a list that +# lost an entry passes every per-name test that only asks about the names it +# still has. +if "$STORE" self >/dev/null 2>&1 || true; then :; fi +xl="$HOME/.xlings/data/xpkgs/xim-x-libcuda-host-link/$SENTINEL_VER/lib" +if command -v xlings >/dev/null 2>&1; then + xlings install "libcuda-host-link@$SENTINEL_VER" -y >"$work/sentinel.log" 2>&1 || true +fi +if [ -d "$xl" ]; then + n=$(ls -1 "$xl" | wc -l) + [ "$n" -ge 2 ] && ok "sentinel $SENTINEL_VER publishes $n sonames" \ + || fail "sentinel publishes $n soname(s); the set is at least 2" + for s in libcuda.so.1 libnvidia-ml.so.1; do + # islink, not exists: a dangling link is the documented self-heal shape + # on a machine with no driver, and the sandbox is such a machine. + [ -L "$xl/$s" ] && ok "sentinel carries $s" || fail "sentinel has no $s" + done +else + skip "B: the sentinel is not installed in this environment ($xl)" +fi + +# --------------------------------------------------------------------------- +section "C. the farm mirrors the sentinel" +# +# THE DEFECT ITSELF. The farm linked one hand-written name while enumerating +# every other directory it draws from, so it carried libcuda.so.1 and not +# libnvidia-ml.so.1, the CUDA adapter did not load, and the program aborted +# with no message. +cat >"$work/mcpp.toml" <"$work/src/main.cpp" <<'CPP' +int main() { return 0; } +CPP +if (cd "$work" && "$STORE" build >"$work/build.log" 2>&1); then + # Searched under the REGISTRY roots rather than all of $HOME: a developer + # home holds tens of gigabytes of packages and the walk costs minutes, + # which reads exactly like a hung verification. + farm="" + for root in "${MCPP_HOME:-$HOME/.mcpp}" "$work/.mcpp" "$HOME/.xlings"; do + [ -d "$root" ] || continue + farm=$(find "$root" -path "*compat-x-sycl-runtime/$COMPAT_VER*/sycl_runtime/lib" \ + -type d 2>/dev/null | head -1) + [ -n "$farm" ] && break + done + if [ -n "$farm" ]; then + for s in libcuda.so.1 libnvidia-ml.so.1; do + [ -L "$farm/$s" ] && ok "farm carries $s" \ + || fail "farm has no $s -- this is mcpp#596" + done + # The denominator: a farm that failed to build is empty, and every + # per-name test above would then have failed for the wrong reason. + n=$(ls -1 "$farm" | wc -l) + [ "$n" -ge 20 ] && ok "farm has $n entries" \ + || fail "farm has only $n entries; the payload half did not build" + else + fail "C: no farm directory for compat:sycl-runtime@$COMPAT_VER" + fi +else + skip "C: the probe project did not build (see $work/build.log; the dpcpp payload is over a gigabyte)" +fi + +# --------------------------------------------------------------------------- +section "D. mcpp reports a dlopen surface it cannot satisfy" +# +# The record rather than the message: a test that greps a warning's wording +# fails the next time the wording improves. Both denominators are asserted for +# the reason they exist -- "no findings" and "nothing was examined" must not +# read the same. +res=$(find "$work" -name resolution.json 2>/dev/null | head -1) +if [ -n "$res" ] && command -v python3 >/dev/null 2>&1; then + python3 - "$res" <<'PY' +import json, sys +doc = json.load(open(sys.argv[1])) +rec = doc.get("runtime", {}).get("dlopen_surface") +if rec is None: + print("ASSERT-FAIL: resolution.json has no runtime.dlopen_surface") + sys.exit(1) +members, walked = rec.get("members", 0), rec.get("walked", 0) +if members <= 0: + print(f"ASSERT-FAIL: dlopen_surface examined {members} members") + sys.exit(1) +print(f"ok: dlopen_surface examined {walked} of {members} members") +missing = [f for f in rec.get("findings", []) if f.get("kind") == "missing"] +for f in missing: + print(f"note: {f['library']} needs {f['soname']} (declared unserved: libOpenCL.so.1)") +bad = [f for f in missing if f.get("soname") == "libnvidia-ml.so.1"] +if bad: + print("ASSERT-FAIL: NVML is still missing from the farm -- mcpp#596") + sys.exit(1) +print("ok: no driver soname is missing from the farm") +PY + [ $? -eq 0 ] || fails=$((fails + 1)) +else + skip "D: no resolution.json (section C did not build) or no python3" +fi + +# --------------------------------------------------------------------------- +printf '\n== summary ==\n' +printf 'assertions failed: %d\n' "$fails" +if [ -n "$skipped" ]; then + printf 'sections NOT RUN:%s\n' "$skipped" + printf 'A pass with sections not run is not a pass for those sections.\n' +fi +exit $((fails > 0)) diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 13da79ea..8dec1f7c 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -273 records. +274 records. ## By subject @@ -32,6 +32,7 @@ Records that declare one. Everything else is listed by date below. ### heterogeneous +- [A dlopen surface no closure walks, and a process with two unwinders](2026-09-09-dlopen-surface-and-two-unwinders.md) — active - [The island boundary's names: one rule for both lanes, and the check that makes it true](2026-09-08-island-boundary-names.md) — active - [Implementation plan: the island boundary's names](2026-09-08-island-boundary-names-implementation-plan.md) — active @@ -44,6 +45,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026-09-09-two-answers-and-two-silences.md) — active +- [A dlopen surface no closure walks, and a process with two unwinders](2026-09-09-dlopen-surface-and-two-unwinders.md) — active - [The documentation as a book: a chapter-by-chapter design](2026-09-08-the-documentation-as-a-book.md) — active - [The island boundary's names: one rule for both lanes, and the check that makes it true](2026-09-08-island-boundary-names.md) — active - [Implementation plan: the island boundary's names](2026-09-08-island-boundary-names-implementation-plan.md) — active diff --git a/CHANGELOG.md b/CHANGELOG.md index 0cd93b6e..2a225995 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,62 @@ ## [Unreleased] +## [2026.9.10.1] - 2026-09-10 + +三条改动来自同一次排查(#596):一个 SYCL 工程构建全绿、运行时以退出码 134 终止且 +不打印任何异常文本。触发原因在生态侧(适配包的 farm 少了一个驱动库),但**它之所以 +以最难排查的形态出现,原因在 mcpp 这一侧**,而且有两条独立的缺口。设计记录见 +`.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md`。 + +### 依赖为 dlopen 发布的那个面,现在会被走一遍 + +运行期闭包检查从产物出发沿 `DT_NEEDED` 走。一个包通过 `runtime.library_dirs` 发布的 +库之所以存在,正是因为有东西要 `dlopen` 它 —— 没有任何链接边指向它,所以它**按构造** +在那条闭包之外,不是被漏掉。实测:一个 25 个成员的 farm 里有两个根本加载不了,而构建 +零诊断。 + +`mcpp build` 之后会单独走这个面,把每个成员自己的 `DT_NEEDED` 按产物真实的搜索路径 +解析一遍,并区分三种读数:解析到(静默)、farm 里存在但链接悬空(机器没装驱动,静默)、 +到处都不存在(打包缺口,警告)。中间那一行是这条检查是**警告而不是错误**的原因。 + +完整结果发布在 `resolution.json` 的 `runtime.dlopen_surface`,含两个分母 +(`members` / `walked`)—— 一个构建失败的 farm 枚举出零个成员,否则「没有发现」与 +「什么都没检查」读起来一模一样。 + +判据要落在**产物真实的搜索路径**上,这一点花了三次修正才对(全部是假阳性,且都要 +先有一个带共享依赖的工程才看得见):产物 `DT_RPATH` 首位的 `$ORIGIN` 不在 +`runtime_search_dirs` 里,而共享依赖就部署在可执行文件旁边;不产出程序的计划没有 +可判的面(适配包自己是 `kind = "lib"`);以及 **SONAME 不是文件名** —— mcpp 链出的是 +`bin/libopencl.so` 而它的 SONAME 是 `libOpenCL.so.1`,别名稍后才出现。 + +### 链接行上出现第二个 C++ 运行时时,进程只保留一个 unwinder + +设备编译器按 libstdc++ 配置的 lane(SYCL、HIP)会把 libstdc++ 放上链接行,而产物 +自己静态链接 libc++。此前的注释断言「可执行文件的静态运行时已经是 local 的」—— +前提对(链接器只导出被加载对象引用到的符号),结论错:**当被加载的对象确实引用它们时, +链接器就把它们放进 `.dynsym`**。实测 89 个导出符号,其中 68 个同时由 libstdc++ 或 +libgcc_s 定义。 + +静态归档只贡献被引用到的成员,所以这种抢占按构造是部分的:libgcc 的 18 个 `_Unwind_*` +入口点里 10 个来自产物、8 个仍在 libgcc_s(含 personality 例程要用的访问器)。 +于是一次抛出被两个 unwinder 分着处理,`__gxx_personality_v0` 越过三帧之上一个本应命中 +的 handler 调用 `__cxa_call_terminate`,而 `__verbose_terminate_handler` 为了打印 +异常类型的那次 rethrow 又终止一次 —— 于是一个字都没印出来。 + +现在这类链接改用 `--unwindlib=libgcc` 并用 `--exclude-libs` 挡住静态归档的导出。 +libgcc_s 本来就在进程里(libstdc++ 需要它),所以这一步只点名一个已有的库。 +链接行上没有第二个运行时的构建一字节不变。 + +实测(同一台机器、同一份源码、两种情形):修改前退出码 134 且无输出;修改后 +`sycl: no usable device: ...` / `device unavailable`,退出码 1;设备可用时两者都是 +`12 24 36 48`。**顺带推翻了示例里一条写下来的结论** —— 设备镜像不匹配的那次抛出 +并非拦不住,它拦不住只是因为 unwinder 是坏的。 + +### 重复符号警告会说出 unwinder 的真实后果 + +该警告此前把后果一律描述为「库自己的那份副本不会被调用」。对 `_Unwind_*` 这一族, +真实后果是异常处理整体失效。重复集合里含该族时,警告会单独说明这一点。 + ## [2026.9.9.1] - 2026-09-09 本次修复的四条缺陷,来源是同一类问题:一个问题被回答了两次,而读答案的地方各读各的; diff --git a/docs/33-authoring-an-adapter.md b/docs/33-authoring-an-adapter.md index 0abe6b20..830b2b89 100644 --- a/docs/33-authoring-an-adapter.md +++ b/docs/33-authoring-an-adapter.md @@ -82,6 +82,44 @@ is then empty and the program reports what it actually found. An adapter that errors on a missing host library turns a supported configuration into a build failure. +## The check mcpp runs on this surface + +The libraries an adapter publishes are reached by `dlopen`, so no link edge +names them and the runtime closure check — which walks `DT_NEEDED` from the +artifact — cannot arrive at them. mcpp walks them separately, after the link, +and reports what it finds as a warning: + +``` +warning: 1 of 13 libraries a dependency published for dlopen cannot be loaded +on this artifact's search path: + libur_adapter_cuda.so.0 needs libnvidia-ml.so.1 +``` + +Three states are separated, and only one of them is reported: + +| The library needs a SONAME that is | Meaning | Reported | +|---|---|---| +| on the artifact's search path | nothing to say | no | +| present in the farm as a dangling link | the machine has no such driver | no | +| absent everywhere | the adapter did not carry it | yes | + +The middle row is why this is a warning and not an error: a dangling link is +the documented shape of a host driver that is not installed, and a check that +failed there would turn a supported configuration into a build failure. + +The full result, including both denominators, is published as +`runtime.dlopen_surface` in `resolution.json`: + +```json +{ "members": 13, "walked": 13, + "findings": [ { "library": "libur_adapter_cuda.so.0", "dir": "...", + "soname": "libnvidia-ml.so.1", "kind": "missing" } ] } +``` + +`members` and `walked` are published even when nothing is reported. A farm that +failed to build enumerates nothing, and "no findings" would otherwise be +indistinguishable from "nothing was examined". + ## Current limitations - **Linux only, by construction.** macOS's dyld and the Windows PE loader have diff --git a/docs/42-heterogeneous-builds.md b/docs/42-heterogeneous-builds.md index 4ef7aa60..f9ba54f2 100644 --- a/docs/42-heterogeneous-builds.md +++ b/docs/42-heterogeneous-builds.md @@ -676,6 +676,29 @@ Which shape to choose is a property of the program, not of mcpp: a seam that swaps an implementation wants link-time selection, and a program that ships to machines it has not seen wants run-time selection. +## One unwinder, when the link carries a second C++ runtime + +A lane whose device compiler is configured against libstdc++ puts libstdc++ on +the link line while the artifact links libc++ statically. Both are then in the +image, and mcpp's duplicate-symbol check reports what they share. + +For most of those symbols the consequence is that one implementation is called +instead of an interchangeable other. For the unwinder it is not. A static +archive contributes only the members something references, so the interposition +is partial by construction: measured on the SYCL lane, ten of libgcc's eighteen +`_Unwind_*` entry points came from the artifact and eight stayed in libgcc_s, +including the accessors a personality routine uses. libstdc++'s personality +then read an LLVM libunwind context through libgcc's accessors, found no +landing pad, and called `std::terminate` past a handler three frames up. The +program was correct until something threw. + +So when the link line names libstdc++ and the toolchain's own library is +libc++, mcpp links the unwinder from libgcc (`--unwindlib=libgcc`) instead of +the payload's `libunwind.a`, and hides the static archives' symbols with +`--exclude-libs`. libgcc_s is already in the process — libstdc++ needs it — so +this names a library rather than adding one, and the C++ runtime stays +embedded. A build with no second runtime on its line is unchanged. + ## Two boundaries worth stating **`--accel` and `--no-accel` are `build`, `run` and `test` options** (run and diff --git a/docs/zh/33-authoring-an-adapter.md b/docs/zh/33-authoring-an-adapter.md index b1dff02f..ea704c92 100644 --- a/docs/zh/33-authoring-an-adapter.md +++ b/docs/zh/33-authoring-an-adapter.md @@ -67,6 +67,40 @@ rpath : …/xim-x-glibc/2.39/lib64:…/xim-x-gcc/…/lib64:$ORIGIN 每一台 CI runner 都是这样的机器。此时农场为空,程序报告它实际找到了什么。一个在宿主库 缺失时报错的适配包,会把一种受支持的配置变成构建失败。 +## mcpp 对这个面的检查与报告 + +适配包发布的库是被 `dlopen` 找到的,没有任何链接边指向它们,因此运行期闭包检查 +—— 它从产物出发沿 `DT_NEEDED` 走 —— 按构造到不了这些库。mcpp 在链接之后单独走一遍 +这个面,并把结果作为警告报出: + +``` +warning: 1 of 13 libraries a dependency published for dlopen cannot be loaded +on this artifact's search path: + libur_adapter_cuda.so.0 needs libnvidia-ml.so.1 +``` + +三种读数被区分开,只有一种被报告: + +| 该库需要的 SONAME | 含义 | 是否报告 | +|---|---|---| +| 在产物的搜索路径上 | 无话可说 | 否 | +| 在农场里存在,但链接悬空 | 这台机器没有这个驱动 | 否 | +| 到处都不存在 | 适配包没有携带它 | 是 | + +中间那一行正是这条检查是警告而不是错误的原因:悬空链接是「宿主驱动尚未安装」的 +既定形状,一条在那里失败的检查会把受支持的配置变成构建失败。 + +完整结果(含两个分母)发布在 `resolution.json` 的 `runtime.dlopen_surface`: + +```json +{ "members": 13, "walked": 13, + "findings": [ { "library": "libur_adapter_cuda.so.0", "dir": "...", + "soname": "libnvidia-ml.so.1", "kind": "missing" } ] } +``` + +即使没有任何发现,`members` 与 `walked` 也会被发布。一个构建失败的农场枚举出零个成员, +否则「没有发现」与「什么都没检查」就读起来一模一样。 + ## 当前边界 - **按构造只适用于 Linux。** macOS 的 dyld 与 Windows 的 PE 加载器没有对应的这一层, diff --git a/docs/zh/42-heterogeneous-builds.md b/docs/zh/42-heterogeneous-builds.md index 584e5f72..e6495b30 100644 --- a/docs/zh/42-heterogeneous-builds.md +++ b/docs/zh/42-heterogeneous-builds.md @@ -562,6 +562,25 @@ sources = ["src/cpu/*.cpp"] 选哪个形态是**程序自己的性质**,不是 mcpp 的:用接缝换实现的程序要链接期选择,而要发到 没见过的机器上去的程序要运行期选择。 +## 链接行上出现第二个 C++ 运行时时,进程只保留一个 unwinder + +一条 lane 的设备编译器若是按 libstdc++ 配置的,链接行上就会出现 libstdc++, +而产物本身静态链接 libc++。两者同时在镜像里,mcpp 的重复符号检查会报出它们共有的符号。 + +对其中大多数符号,后果只是「调用了另一份可互换的实现」。对 unwinder 不是。 +静态归档只会贡献被引用到的成员,所以这种抢占**按构造是部分的**:在 SYCL lane 上实测, +libgcc 的 18 个 `_Unwind_*` 入口点里有 10 个来自产物、8 个仍在 libgcc_s, +其中包括 personality 例程要用的那几个访问器。于是 libstdc++ 的 personality +拿 libgcc 的访问器去读一个 LLVM libunwind 的 context,找不到 landing pad, +越过三帧之上一个本应命中的 handler 直接 `std::terminate`。 +在有东西抛出之前,这个程序一直是正确的。 + +因此,当链接行上出现 libstdc++ 而工具链自带的标准库是 libc++ 时, +mcpp 改从 libgcc 取 unwinder(`--unwindlib=libgcc`),不再链载荷的 `libunwind.a`, +并用 `--exclude-libs` 把静态归档的符号挡在动态符号表之外。libgcc_s 本来就在进程里 +—— libstdc++ 需要它 —— 所以这一步只是点名一个已有的库而不是新增一个, +C++ 运行时仍然是内嵌的。链接行上没有第二个运行时的构建一字节不变。 + ## 两条值得写明的边界 **`--accel` 与 `--no-accel` 是 `build`、`run`、`test` 三者的选项**(run 与 test 自 2026.9.5.2 起),与 `--target`、`--profile` 同级;`pack` 与其它构建输入一样从 manifest 读 `[build] accel`。它起初只挂在 `build` 上,实测的后果是一个工程的 CPU-only 变体能构建却不能运行:`mcpp build --no-accel` 产出了它,而 `mcpp run` 交回的是设备构建。 diff --git a/examples/09-heterogeneous/sycl/README.md b/examples/09-heterogeneous/sycl/README.md index f3600225..d5c5a260 100644 --- a/examples/09-heterogeneous/sycl/README.md +++ b/examples/09-heterogeneous/sycl/README.md @@ -73,8 +73,18 @@ without them and names the line to add. ## Two C++ runtimes, and why the seam is not optional here `libsycl.so` is compiled against libstdc++ while an mcpp artifact links libc++, -so both are in the image. mcpp's duplicate-symbol check reports the unwinder -symbols they share, and the warning is correct. +so both are in the image. + +The unwinder is the one part of that seam a `catch` cannot police, because +nothing in this source reaches it: an exception is raised through whichever +`_Unwind_*` the process resolved, and a static archive contributes only the +members something referenced. On this lane ten of libgcc's eighteen entry +points came from the artifact and eight from libgcc_s, so libstdc++'s +personality routine read an LLVM libunwind context through libgcc's accessors +and every handler below was skipped. mcpp gives such a link one unwinder +(`--unwindlib=libgcc`, since the process already has libstdc++'s) and hides the +static archives' symbols, so the duplicate-symbol warning no longer fires here +and the `catch` blocks below do what they say. Nothing may cross the seam. The island catches its own `sycl::exception` and returns a code, because the runtime that threw it is not the one the caller @@ -91,10 +101,17 @@ Making that promise true took three things, and only two of them are a `catch`: without one gets the default handler, and the default handler calls `std::terminate` — which no `catch` can intercept, since it never travels as an exception through this frame; -* and one failure remains outside both. A build compiled to SPIR-V, run against - a back end that does not consume it, throws from inside the SYCL scheduler. - That is why this manifest names the device, and why `mcpp.rules.sycl` warns at - build time when an `accel` names `sycl` and no device. +* and the third is not a `catch` at all: **one unwinder in the process**. A + build whose image does not match the device throws from inside the SYCL + scheduler, and the sentence that used to stand here said that failure was + outside both catches. It was outside them because no catch worked. Measured + on one machine, same source, same device: with two unwinders, exit 134 and no + output; with one, `sycl: The program was built for 1 devices` followed by + `device unavailable`, exit 1. + +The manifest still names the device, and `mcpp.rules.sycl` still warns when an +`accel` names `sycl` and no device: an image compiled for the device is the +point of an ahead-of-time build. ## Running it diff --git a/examples/09-heterogeneous/sycl/app/mcpp.toml b/examples/09-heterogeneous/sycl/app/mcpp.toml index 3fc75d80..97ac1963 100644 --- a/examples/09-heterogeneous/sycl/app/mcpp.toml +++ b/examples/09-heterogeneous/sycl/app/mcpp.toml @@ -30,8 +30,14 @@ plugins = { version = "0.5.2", features = ["rules-sycl", "tools-island"], host-m # SYCL runtime's CUDA back end dlopens the driver, so `compat:sycl-runtime` # carries that hop itself from 2026.09.07 on. A project that writes SYCL does # not have to know CUDA is underneath it. +# +# 2026.09.10 is the version at which it carries the WHOLE hop. Until then it +# linked one driver library by name, and the adapter needs two: the second was +# missing, the CUDA back end did not load, and the program aborted with no +# message (#596). The farm now mirrors what the driver sentinel publishes +# rather than naming a file. [dependencies.compat] -sycl-runtime = "2026.09.07" +sycl-runtime = "2026.09.10" # NO [xlings.workspace]. `mcpp.rules.sycl` declares all five payloads this lane # needs, each closing one hole the host would otherwise fill: `dpcpp` (the diff --git a/examples/09-heterogeneous/sycl/app/src/kernels/saxpy.sycl b/examples/09-heterogeneous/sycl/app/src/kernels/saxpy.sycl index d61ce400..2ad06b15 100644 --- a/examples/09-heterogeneous/sycl/app/src/kernels/saxpy.sycl +++ b/examples/09-heterogeneous/sycl/app/src/kernels/saxpy.sycl @@ -23,12 +23,27 @@ // // It catches the synchronous half (no usable device, a rejected submit) and // installs a handler for the asynchronous half, which SYCL otherwise delivers -// to a default handler that calls `std::terminate`. What it cannot catch is a -// missing device image: a build compiled to SPIR-V, run against a back end -// that does not consume SPIR-V, throws from inside the scheduler -// (`ProgramManager::getDeviceImage`) in neither of those two paths. That is -// why the manifest names the device -- and why `mcpp.rules.sycl` warns at -// build time when it does not. +// to a default handler that calls `std::terminate`. +// +// A MISSING DEVICE IMAGE IS CAUGHT TOO, AND THE SENTENCE THAT USED TO SAY +// OTHERWISE WAS DESCRIBING A BROKEN UNWINDER. It read: what this cannot catch +// is a missing device image, because it throws from inside the scheduler in +// neither of those two paths. That was measured on a build whose two C++ +// runtimes had split the unwinder between them, so NO handler in this file ran +// -- including the one three frames above the throw. With one unwinder in the +// process (mcpp 2026.9.10.1), the same build compiled for the wrong +// architecture prints +// +// sycl: The program was built for 1 devices +// device unavailable +// +// and exits 1. Measured both ways on one machine: exit 134 and silence before, +// exit 1 and the text above after. +// +// The manifest still names the device, and `mcpp.rules.sycl` still warns when +// it does not, for the reason that has not changed: an image compiled for the +// device is the point of an ahead-of-time build. What changed is that failing +// to have one is now something this island can report. #include #include #include diff --git a/mcpp.toml b/mcpp.toml index ded70a15..70e5074e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.9.1" +version = "2026.9.10.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index e30a29e8..376ad844 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.9.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.10.1"; } // namespace mcpp diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index f100f225..52ca0d69 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -269,6 +269,17 @@ struct MechanismInput { // `linkage = "static"` — the libc axis. On PE it shares the one `-static` // spelling with the C++ runtime axis, so the table has to see it. bool fullStaticLibc = false; + // Does this link line also name a C++ runtime that is NOT the toolchain's? + // + // Today that means a libc++ toolchain whose line carries libstdc++, + // which is what the SYCL and HIP rule packages produce: the device half is + // compiled by a second compiler configured against libstdc++, so the + // artifact links libc++ statically AND loads libstdc++.so at run time. + // + // It changes two things below, and both are corrections of an assumption + // that held only while no such line existed. See `hide_static_cxx_runtime` + // for the symbol half and the libc++ ELF branch for the unwinder half. + bool foreignCxxRuntime = false; // Already-escaped archive paths for the explicit-archive mechanisms. // Empty string = that archive is not available on this toolchain. std::string libcxxArchive; @@ -362,13 +373,26 @@ inline bool is_libcxx(std::string_view id) { return id == "libc++"; } // Keep a statically linked standard library OUT of a shared object's dynamic // symbol table. // -// Only a SHARED LIBRARY needs this, and only when it actually embedded the +// A SHARED LIBRARY always needs this, and only when it actually embedded the // runtime — which after `default_contract` happens on ELF exclusively through -// an explicit `cxx_runtime = { shared = "self-contained" }`. An executable's -// static libstdc++ is already local (ld exports only what a loaded object -// references, and mcpp passes no `-rdynamic`); a .so exports every global it -// defines, which is how a pure-C compat package came to publish 777 GLOBAL -// libstdc++ definitions and become the executable's de-facto C++ runtime. +// an explicit `cxx_runtime = { shared = "self-contained" }`. A .so exports +// every global it defines, which is how a pure-C compat package came to +// publish 777 GLOBAL libstdc++ definitions and become the executable's +// de-facto C++ runtime. +// +// AN EXECUTABLE NEEDS IT WHEN A FOREIGN C++ RUNTIME IS ON THE LINE, and the +// sentence that used to be here — "an executable's static libstdc++ is already +// local (ld exports only what a loaded object references, and mcpp passes no +// `-rdynamic`)" — was a correct premise with a wrong conclusion. The clause in +// the parentheses is the whole mechanism: when a loaded object DOES reference +// them, the linker puts them in `.dynsym`. Measured on a SYCL artifact +// (mcpp#596), which links libc++ statically and loads libstdc++.so: 89 +// exported symbols, 68 of them also defined by libstdc++ or libgcc_s. The +// executable is searched first, so libstdc++'s own code called libc++abi's +// `std::exception::what`, libc++'s `std::runtime_error` constructors ran on +// objects libstdc++ would later destroy, and ten of libgcc_s's eighteen +// unwinder entry points were answered by the executable's libunwind while +// eight were not. // // It does NOT hide the weak/COMDAT template instantiations the library's own // code emits, and must not: unifying those across the process is the intended @@ -379,9 +403,9 @@ inline bool is_libcxx(std::string_view id) { return id == "libc++"; } // Archive BASENAMES — that is what `--exclude-libs` matches, and GNU ld and // lld agree on it. Listed by name rather than `ALL` so a user's own static // library linked into their .so keeps its exports. -std::string hide_static_cxx_runtime(Role role, +std::string hide_static_cxx_runtime(Role role, bool foreignCxxRuntime, std::initializer_list archives) { - if (role != Role::SharedLibrary) return {}; + if (role != Role::SharedLibrary && !foreignCxxRuntime) return {}; std::string out; for (auto archive : archives) { out += " -Wl,--exclude-libs,"; @@ -591,7 +615,7 @@ Mechanism resolve(const MechanismInput& in) { if (m.effective == Contract::SelfContained) { m.unitFlags = " -static-libstdc++"; m.unitFlags += detail::hide_static_cxx_runtime( - in.role, {"libstdc++.a"}); + in.role, in.foreignCxxRuntime, {"libstdc++.a"}); } // ToolchainCoupled and HostCoupled are the same emission on ELF // (no flag); they differ in the rpath the link already carries, @@ -621,11 +645,34 @@ Mechanism resolve(const MechanismInput& in) { m.unitFlags = " -nostdlib++ " + in.libcxxArchive + " " + in.libcxxAbiArchive; m.unitFlags += detail::hide_static_cxx_runtime( - in.role, {"libc++.a", "libc++abi.a"}); - if (!in.libunwindArchive.empty()) { + in.role, in.foreignCxxRuntime, {"libc++.a", "libc++abi.a"}); + if (in.foreignCxxRuntime) { + // ONE UNWINDER PER PROCESS. + // + // libgcc_s is in this process either way: libstdc++.so needs + // it, so naming it here adds no loaded object -- measured, the + // NEEDED set gains the name and nothing else. What it removes + // is the second unwinder. Linking libunwind.a instead pulls in + // only the archive members something references, so ten of + // libgcc's eighteen entry points came from the executable and + // eight stayed in libgcc_s; libstdc++'s personality routine + // then read an LLVM libunwind context through libgcc's + // accessors, found no landing pad, and terminated past a + // handler that should have run (mcpp#596). + // + // `--unwindlib=libgcc` LAST WINS over the payload cfg file's + // `--unwindlib=libunwind`, which is why this is an addition + // rather than an edit of that file: the cfg is the default for + // every link, and only this line has the second runtime on it. + // + // NOT a degradation of the contract. The C++ runtime is still + // embedded; the unwinder was never the artifact's own here, + // because the process already had libstdc++'s. + m.unitFlags += " --unwindlib=libgcc"; + } else if (!in.libunwindArchive.empty()) { m.unitFlags += " " + in.libunwindArchive; m.unitFlags += detail::hide_static_cxx_runtime( - in.role, {"libunwind.a"}); + in.role, in.foreignCxxRuntime, {"libunwind.a"}); } else { m.degraded = true; // effective stays SelfContained: the C++ // runtime IS embedded; the unwinder is not diff --git a/src/build/flags.cppm b/src/build/flags.cppm index e89b54f9..7c2a64c1 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -1028,6 +1028,23 @@ CompileFlags compute_flags(const BuildPlan& plan) { mi.mingw = isMingwTc; mi.macosFloor = !macosDeploymentTarget.empty(); mi.format = format; + // A SECOND C++ RUNTIME ON THIS LINE. + // + // Read off the link flags rather than inferred, because the only way + // one gets there is that something wrote it: a rule package whose + // second compiler is configured against libstdc++ emits + // `-l:libstdc++.so.6` through `mcpp::link_lib`, which lands in + // `bc.ldflags`. `mcpp.rules.sycl` and `mcpp.rules.hip` are the two + // that do today. + // + // The substring is `stdc++`, which covers `-lstdc++`, + // `-l:libstdc++.so.6` and a spelled-out path. It cannot match this + // toolchain's own runtime: the predicate requires libc++, and a + // libstdc++ toolchain never reaches the branch that reads this. + mi.foreignCxxRuntime = + caps.stdlib_id == "libc++" + && std::ranges::any_of(bc.ldflags, [](std::string_view flag) { + return flag.find("stdc++") != std::string_view::npos; }); // Bare metal short-circuits the whole table: the archives found below // are the HOST's, and linking them into a riscv64 image fails with // "incompatible with elf64lriscv". See MechanismInput::freestanding. diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index d5df6b3a..4ff138ce 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -3073,6 +3073,46 @@ std::expected NinjaBackend::build(const BuildPlan& plan explanation)); } } + // THE SURFACE THE ARTIFACT WALK CANNOT REACH. + // + // Everything above follows DT_NEEDED from an artifact. A library a + // dependency published through `runtime.library_dirs` is there because + // something will `dlopen` it, so no link-time edge names it and the + // walk cannot arrive. Reported after the artifacts and never as a + // failure: see `check_dlopen_surface` for why advisory is the right + // severity. + { + auto surface = + mcpp::build::runtime_validation::check_dlopen_surface(plan); + std::vector gaps; + for (auto const& finding : surface.findings) { + if (finding.dangling) continue; // the machine's answer + gaps.push_back(std::format("{} needs {}", + finding.member.filename().string(), + finding.soname)); + } + if (!gaps.empty()) { + std::string detail; + for (auto const& gap : gaps) detail += "\n " + gap; + mcpp::ui::warning(std::format( + "{} of {} librar{} a dependency published for dlopen cannot " + "be loaded on this artifact's search path:{}\n" + // The plural agrees with the DENOMINATOR, which is what the + // sentence is about; agreeing with the numerator produced + // "1 of 16 library". + " These are reached by dlopen, so no link edge names them " + "and the closure check above cannot see them. The program " + "links and runs until something asks for one, and then the " + "back end behind it is simply absent.\n" + " Fix: the package that published the directory has to " + "carry the missing library too, or declare that it does " + "not serve it.\n" + " Record: `runtime.dlopen_surface` in resolution.json " + "({} of {} examined).", + gaps.size(), surface.members, surface.members == 1 ? "y" : "ies", + detail, surface.walked, surface.members)); + } + } if (!runtimeFailure.empty()) { return std::unexpected(BuildError{ "runtime closure validation failed (proven Linux ELF defect)", diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index 074b76ed..b8562890 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -158,6 +158,21 @@ std::vector check_symbol_provision(const mcpp::build::BuildPlan& plan, const ArtifactSnapshot& before); +// Walk the libraries this build's dependencies published for `dlopen`. +// +// A SEPARATE ENTRY POINT for the reason the one above is: its object is not an +// artifact. `validate_changed_artifacts` answers a question per image and +// memoises on that image's stat; this one answers a question about +// DIRECTORIES, which no artifact's stat describes, and it holds for a build +// that linked nothing new. +// +// ADVISORY, never blocking. A farm legitimately holds host-driver links that +// dangle on a machine with no driver, and turning a correct CPU-only +// configuration into a failed build would be a worse defect than the one this +// reports. +mcpp::platform::elf::DlopenSurfaceReport +check_dlopen_surface(const mcpp::build::BuildPlan& plan); + } // namespace mcpp::build::runtime_validation namespace mcpp::build::runtime_validation { @@ -300,6 +315,7 @@ std::string cache_key(const mcpp::build::BuildPlan& plan, // verdicts. One authoritative writer, one published view. constexpr std::string_view kLoaderTagsRecord = "loader_tags"; constexpr std::string_view kSymbolProvisionRecord = "symbol_provision"; +constexpr std::string_view kDlopenSurfaceRecord = "dlopen_surface"; // WHAT INVALIDATES A STORED VERDICT BESIDES THE ARTIFACT ITSELF. // @@ -403,6 +419,7 @@ void persist_post_link(const mcpp::build::BuildPlan& plan, if (keyMoved) { doc.erase(std::string(kLoaderTagsRecord)); doc.erase(std::string(kSymbolProvisionRecord)); + doc.erase(std::string(kDlopenSurfaceRecord)); } if (keyMoved || doc.value(std::string(name), nlohmann::json::array()) != entries) { doc["post_link_key"] = std::string(key); @@ -1117,4 +1134,108 @@ latest_stored_verdict(const std::filesystem::path& targetRoot) { return summary; } +mcpp::platform::elf::DlopenSurfaceReport +check_dlopen_surface(const mcpp::build::BuildPlan& plan) { + mcpp::platform::elf::DlopenSurfaceReport report; + if constexpr (!mcpp::platform::is_linux) return report; + + // THE SAME APPLICABILITY THE ARTIFACT VERDICT HAS, and for the same + // reason. Under a non-hermetic binding the host loader also consults + // `ld.so.cache`, which mcpp deliberately does not parse, so "not on the + // path mcpp computed" is not evidence of anything. `allow_host_libs` is + // the user's statement that resolution is theirs to arrange. + if (!plan.runtimeBinding.hermetic() || host_libs_allowed(plan)) + return report; + + auto searchDirs = runtime_search_dirs(plan); + + // THE ARTIFACT'S OWN DIRECTORY IS PART OF THE PATH AND IS NOT IN THAT LIST. + // + // Every artifact mcpp links carries `$ORIGIN` first in its DT_RPATH, and a + // dependency built as a shared library is deployed BESIDE it. So a farm + // member needing that library resolves at run time and would be reported + // as a packaging gap here. + // + // Measured: `compat:sycl-runtime` declares `compat:opencl`, whose + // `libOpenCL.so.1` lands in the consumer's `bin/` next to the executable — + // exactly what `libur_adapter_opencl.so.0` needs — and this check named it + // missing until the directory was added. `runtime_search_dirs` cannot + // carry it: `$ORIGIN` is a property of each artifact, not of the plan. + // A PLAN THAT PRODUCES NO PROGRAM HAS NO dlopen SURFACE. + // + // The surface is reached from a PROCESS, and only an executable starts + // one. A plan that produces archives, or a dependency's shared library on + // the way to something else, has no search path of its own to judge + // against -- the question belongs to whatever eventually runs, which has + // its own plan and its own answer. + // + // Measured, and both halves of that sentence were needed. The adapter + // package is `kind = "lib"`, so it produces an archive. And `mcpp test` + // drives the backend twice: the first pass links the dependency's shared + // library and nothing else, and reported `libur_adapter_opencl.so.0 needs + // libOpenCL.so.1` -- the very library it was in the middle of producing. + const bool producesAProgram = std::ranges::any_of( + plan.linkUnits, [](auto const& unit) { + return unit.kind == mcpp::build::LinkUnit::Binary + || unit.kind == mcpp::build::LinkUnit::TestBinary; + }); + if (!producesAProgram) return report; + const auto artifacts = snapshot_link_artifacts(plan); + if (artifacts.empty()) return report; + + for (auto const& [artifact, stamp] : artifacts) { + auto dir = artifact.parent_path(); + if (dir.empty() || std::ranges::find(searchDirs, dir) != searchDirs.end()) + continue; + searchDirs.push_back(dir); + } + + // The SONAMEs this build produces, read from the objects rather than from + // their filenames. See `inspect_dlopen_surface` for why a filename search + // is not enough while the build is still running. + std::vector produced; + for (auto const& [artifact, stamp] : artifacts) { + auto facts = mcpp::platform::elf::inspect_elf_runtime(artifact); + if (facts && !facts->soname.empty()) produced.push_back(facts->soname); + } + + report = mcpp::platform::elf::inspect_dlopen_surface( + plan.depRuntimeLibraryDirs, plan.runtimeBinding, searchDirs, produced); + + // THE RECORD, AND BOTH DENOMINATORS IN IT. + // + // A warning scrolls past; `resolution.json` is the documented place to + // look (docs/05) and is what a test can assert a FIELD of rather than a + // substring of a message. `members` and `walked` are published even when + // there is nothing to report, because "no findings" and "nothing was + // examined" are the two readings this repository has most often confused: + // a surface that failed to build enumerates zero members and every + // per-member test then passes. + nlohmann::json entries = nlohmann::json::array(); + for (auto const& finding : report.findings) { + entries.push_back({ + // The name the loader asks for, and the directory that published + // it. `lexically_relative` rather than `std::filesystem::relative`: + // the latter canonicalises, and every member of such a directory + // is a symlink into a payload -- so the published record named the + // payload's file and lost which farm entry could not be satisfied. + {"library", finding.member.filename().string()}, + {"dir", finding.member.parent_path() + .lexically_relative(plan.outputDir) + .lexically_normal().generic_string()}, + {"soname", finding.soname}, + // `dangling` is the machine's answer (no driver installed); + // anything else is a packaging gap in the publishing package. + {"kind", finding.dangling ? "dangling" : "missing"}, + }); + } + nlohmann::json record = { + {"members", report.members}, + {"walked", report.walked}, + {"findings", std::move(entries)}, + }; + persist_post_link(plan, kDlopenSurfaceRecord, post_link_key(plan), record); + return report; +} + } // namespace mcpp::build::runtime_validation diff --git a/src/build/symbol_provision.cppm b/src/build/symbol_provision.cppm index 624de9e9..7df7d8b7 100644 --- a/src/build/symbol_provision.cppm +++ b/src/build/symbol_provision.cppm @@ -242,7 +242,42 @@ std::string Report::explain(std::string_view artifact) const { " The executable is searched first, so the copy inside it wins for\n" " every symbol both provide — the library's own copy is never called,\n" " and code inside that library now runs against a build it was not\n" - " linked against.\n" + " linked against.\n"; + + // THE UNWINDER FAMILY IS NOT ONE MORE DUPLICATE SYMBOL. + // + // For every other name the sentence above is the whole consequence: one + // implementation is called instead of another, and the two are usually + // interchangeable. For `_Unwind_*` the consequence is that a throw is + // processed by TWO unwinders and no `catch` runs. + // + // A static archive contributes only the members something references, so + // the interposition is PARTIAL by construction. Measured on a SYCL + // artifact (mcpp#596): 10 of libgcc_s's 18 entry points came from the + // executable's libunwind and 8 stayed in libgcc_s, including the context + // accessors — so libstdc++'s personality routine read an LLVM libunwind + // `_Unwind_Context` through libgcc's `_Unwind_GetIPInfo`, recovered a + // meaningless IP, found no landing pad, and `__cxa_call_terminate` ran + // past a handler three frames up. The program aborted with exit 134 and + // printed nothing, because `__verbose_terminate_handler` rethrows to name + // the exception's type and that rethrow terminated as well. + // + // Said separately rather than folded into the list below, because none of + // the three ways out addresses it: the fix is one unwinder in the process, + // which is a link-line question rather than a packaging one. + if (std::ranges::any_of(conflicts, [](auto const& c) { + return c.name.starts_with("_Unwind_"); })) { + body += + " These include the unwinder's entry points (`_Unwind_*`), and\n" + " for those the consequence is stronger: an exception is then\n" + " raised by one unwinder and inspected by the other, no `catch`\n" + " matches, and the program calls std::terminate past a handler\n" + " that should have run. A static archive contributes only the\n" + " members something references, so the split is partial and the\n" + " program is correct until something throws.\n"; + } + + body += " Ways out, in the order they apply:\n" " 1. stop one side from providing it — usually the package that\n" " ships a copy of a library the graph already builds;\n" diff --git a/src/runtime/elf.cppm b/src/runtime/elf.cppm index 5c13b2e8..c5fed09d 100644 --- a/src/runtime/elf.cppm +++ b/src/runtime/elf.cppm @@ -193,6 +193,55 @@ RuntimeVerdict validate_runtime_artifact( const RuntimeResolution& resolution, bool hostLibsAllowed = false); +// THE SURFACE NOTHING ELSE WALKS. +// +// `resolve_runtime_closure` above is seeded with the artifact and follows +// DT_NEEDED. A library a package published through `runtime.library_dirs` +// exists precisely because something will `dlopen` it, so no link-time edge +// names it and it is outside that closure BY CONSTRUCTION, not by oversight. +// +// Measured (mcpp#596): a farm of twenty-five libraries, two of which could not +// load at all -- one needing `libnvidia-ml.so.1`, one needing +// `libOpenCL.so.1`, neither on the search path -- while the build reported no +// diagnostic, because nothing had asked. The program then aborted at run time +// with no message. +// +// mcpp is the only component that can answer this: it computed the whole +// search path, and under a hermetic binding nothing else will be consulted. +// The check is provider-agnostic -- it knows nothing of SYCL, CUDA or Unified +// Runtime, and must not, for the reason `test_runtime_contract` states. +struct DlopenSurfaceFinding { + std::filesystem::path member; // the library that cannot be satisfied + std::string soname; // what it needs + // Present as an entry but pointing nowhere. The host-driver sentinels + // create deliberately dangling links so that installing a driver later + // self-heals every consumer, so this is the MACHINE's answer and never an + // error. `Missing` is the packaging gap. + bool dangling = false; +}; + +struct DlopenSurfaceReport { + // THE DENOMINATORS. "No findings" and "nothing was examined" are the two + // readings that must not be spelled the same: a directory that failed to + // build enumerates nothing, and every per-member test then passes. + std::size_t members = 0; // versioned sonames found in the directories + std::size_t walked = 0; // whose own DT_NEEDED was read + std::vector findings; +}; + +// +// `alsoProvided` names SONAMEs this build produces itself. An artifact carries +// its SONAME in the object rather than in its filename -- mcpp links +// `bin/libopencl.so` whose SONAME is `libOpenCL.so.1`, and the alias under the +// SONAME is created by a later step -- so a filename search reports the +// library as missing while it is being produced. Measured: `mcpp test` calls +// this twice and only the second call saw the alias. +DlopenSurfaceReport inspect_dlopen_surface( + std::span surfaceDirs, + const mcpp::platform::runtime::RuntimeBinding& binding, + std::span searchDirs, + std::span alsoProvided = {}); + } // namespace mcpp::platform::elf namespace mcpp::platform::elf { @@ -1258,4 +1307,88 @@ RuntimeVerdict validate_runtime_artifact( return verdict; } +DlopenSurfaceReport inspect_dlopen_surface( + std::span surfaceDirs, + const mcpp::platform::runtime::RuntimeBinding& binding, + std::span searchDirs, + std::span alsoProvided) { + DlopenSurfaceReport report; + + // A versioned SONAME is what a farm links and what dlopen asks for; an + // unversioned `libfoo.so` in such a directory is a LINK-time name and is + // not part of this surface. `-gdb.py` sidecars sit beside the payload's + // libraries and match a looser test. + auto versioned = [](const std::string& name) { + auto so = name.find(".so."); + if (so == std::string::npos || name.size() < so + 5) return false; + if (name.ends_with(".py")) return false; + return name[so + 4] >= '0' && name[so + 4] <= '9'; + }; + + // Every entry in every surface directory, indexed by SONAME, so that a + // member needing another member is answered from the surface itself. + std::map entries; + std::error_code ec; + for (auto const& dir : surfaceDirs) { + if (dir.empty() || !std::filesystem::is_directory(dir, ec)) continue; + for (auto const& item : std::filesystem::directory_iterator(dir, ec)) { + auto name = item.path().filename().string(); + if (!versioned(name)) continue; + entries.emplace(name, item.path()); + } + } + + // ONE LIBRARY, NOT ITS TWO NAMES. A farm carries `libfoo.so.N` and + // `libfoo.so.N.M.P` as two links to one file, so walking the entries + // reports every finding twice and calls thirteen libraries twenty-five. + // Keyed by the file both resolve to; the SHORTEST name is kept, which is + // the one a dlopen asks for. + std::map> + libraries; + for (auto const& [name, path] : entries) { + auto real = std::filesystem::weakly_canonical(path, ec); + if (ec) { real = path; ec.clear(); } + auto it = libraries.find(real); + if (it == libraries.end()) libraries.emplace(real, std::pair{name, path}); + else if (name.size() < it->second.first.size()) + it->second = {name, path}; + } + report.members = libraries.size(); + + for (auto const& [name, path] : std::views::values(libraries)) { + // A dangling entry is the sentinel's self-heal shape: the package did + // its part and the target is the machine's answer. Nothing to read. + if (!std::filesystem::exists(path, ec)) continue; + auto facts = inspect_elf_runtime(path); + if (!facts) continue; // not an ELF this reader understands; a + // statement about the CHECK, not the surface + ++report.walked; + for (auto const& soname : facts->needed) { + // Resolved the way the loader will resolve it, from this member's + // own tags plus the search path the artifact actually carries. + if (detail::resolve_needed(soname, *facts, binding, searchDirs, {})) + continue; + // A SONAME this build produces under another filename. + if (std::ranges::find(alsoProvided, soname) != alsoProvided.end()) + continue; + // DANGLING IS A PROPERTY OF THE FILE, NOT OF THE NAME. An entry + // that exists and still did not resolve is a third thing -- an + // unreadable file, a directory the search does not reach -- and + // calling it "the machine has no driver" would silence it. + auto entry = entries.find(soname); + const bool dangling = entry != entries.end() + && !std::filesystem::exists(entry->second, ec); + report.findings.push_back({ + .member = path, + .soname = soname, + .dangling = dangling, + }); + } + } + std::ranges::sort(report.findings, [](auto const& a, auto const& b) { + return std::tie(a.member, a.soname) < std::tie(b.member, b.soname); + }); + return report; +} + } // namespace mcpp::platform::elf diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 12d4a5c2..83ede0dc 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -670,3 +670,72 @@ TEST(Distribution, FormatUsesTheFallbackOnlyWhenTheTripleSaysNothing) { EXPECT_EQ(dist::format_for("", dist::Format::Pe), dist::Format::Pe); EXPECT_EQ(dist::format_for("nonsense", dist::Format::Elf), dist::Format::Elf); } + +// --------------------------------------------------------------------------- +// A SECOND C++ RUNTIME ON THE LINE (mcpp#596). +// +// A lane whose device compiler is configured against libstdc++ puts libstdc++ +// on the link line while the artifact links libc++ statically. Two things then +// have to change, and neither is a preference: +// +// * one unwinder. Linking libunwind.a pulls in only the archive members +// something referenced, so the interposition is partial by construction -- +// measured, ten of libgcc's eighteen entry points came from the artifact +// and eight stayed in libgcc_s, including the accessors libstdc++'s +// personality routine calls. It read an LLVM libunwind context through +// libgcc's accessors and terminated past a matching handler. +// * the static archives' symbols hidden. The linker exports what a loaded +// object references, and a loaded libstdc++ references them: measured, 89 +// exported symbols of which 68 were also defined by libstdc++ or libgcc_s. +// +// The byte-level assertions are the point, as they are for every other cell in +// this table: a build with no second runtime on its line must be unchanged. + +TEST(Distribution, LinuxLibcxxWithAForeignRuntimeTakesOneUnwinder) { + dist::MechanismInput in; + in.format = dist::Format::Elf; + in.stdlibId = "libc++"; + in.requested = dist::Contract::SelfContained; + in.libcxxArchive = "/tc/libc++.a"; + in.libcxxAbiArchive = "/tc/libc++abi.a"; + in.libunwindArchive = "/tc/libunwind.a"; + + // Unchanged when nothing else is on the line. Byte-for-byte the string the + // cell above asserts. + auto alone = dist::resolve(in); + EXPECT_EQ(alone.unitFlags, + " -nostdlib++ /tc/libc++.a /tc/libc++abi.a /tc/libunwind.a"); + + in.foreignCxxRuntime = true; + auto shared = dist::resolve(in); + // The contract is NOT degraded: the C++ runtime is still embedded, and + // libgcc_s is in the process either way because libstdc++ needs it. + EXPECT_EQ(shared.effective, dist::Contract::SelfContained); + EXPECT_FALSE(shared.degraded); + EXPECT_TRUE(shared.diagnostic.empty()); + EXPECT_EQ(shared.unitFlags, + " -nostdlib++ /tc/libc++.a /tc/libc++abi.a" + " -Wl,--exclude-libs,libc++.a -Wl,--exclude-libs,libc++abi.a" + " --unwindlib=libgcc"); + // The payload's unwinder archive is NOT on the line: linking it is what + // creates the second unwinder. + EXPECT_EQ(shared.unitFlags.find("libunwind.a"), std::string::npos); +} + +// A shared library already hid these archives, and that path is untouched: the +// widened guard adds executables, it does not change what a .so emits. +TEST(Distribution, ASharedLibraryStillHidesTheArchivesWithoutASecondRuntime) { + dist::MechanismInput in; + in.format = dist::Format::Elf; + in.stdlibId = "libc++"; + in.role = dist::Role::SharedLibrary; + in.requested = dist::Contract::SelfContained; + in.libcxxArchive = "/tc/libc++.a"; + in.libcxxAbiArchive = "/tc/libc++abi.a"; + in.libunwindArchive = "/tc/libunwind.a"; + auto m = dist::resolve(in); + EXPECT_EQ(m.unitFlags, + " -nostdlib++ /tc/libc++.a /tc/libc++abi.a" + " -Wl,--exclude-libs,libc++.a -Wl,--exclude-libs,libc++abi.a" + " /tc/libunwind.a -Wl,--exclude-libs,libunwind.a"); +} diff --git a/tests/unit/test_elf_runtime.cpp b/tests/unit/test_elf_runtime.cpp index f3e3e328..6bbfe8ad 100644 --- a/tests/unit/test_elf_runtime.cpp +++ b/tests/unit/test_elf_runtime.cpp @@ -694,3 +694,99 @@ TEST(ElfRuntime, ThisBinaryExportsNothingOfItsOwn) { return s; }(); } + +// THE SURFACE A DEPENDENCY PUBLISHES FOR dlopen, WHICH NO ARTIFACT WALK REACHES. +// +// `resolve_runtime_closure` is seeded with the artifact and follows DT_NEEDED. +// A library in a package's `runtime.library_dirs` is there because something +// will dlopen it, so nothing names it and it is outside that closure by +// construction. mcpp#596 is one such library needing a soname no directory on +// the search path carried, on a build that reported no diagnostic at all. +// +// Three states, and the difference between the second and the third is the +// whole reason this is not "dlopen everything and fail on error": a surface +// legitimately holds host-driver links that dangle on a machine with no +// driver, and a check that failed there would turn a correct CPU-only +// configuration into a failed build. +TEST(DlopenSurface, SeparatesAPackagingGapFromAMachineWithoutTheDriver) { + if constexpr (!mcpp::platform::is_linux) + GTEST_SKIP() << "ELF/glibc runtime physics only apply on Linux"; + Tmp t; + auto payload = t.path / "store"; + auto glibc = payload / "2.44" / "lib64"; + auto farm = t.path / "farm"; + std::filesystem::create_directories(glibc); + std::filesystem::create_directories(farm); + + write_elf_fixture(glibc / "libc.so.6", {.needed = {}, .runpath = glibc.string()}); + + // Resolved: needs only what the surface itself carries. + write_elf_fixture(farm / "libgood.so.1", + {.needed = {"libc.so.6"}, .runpath = glibc.string()}); + // The packaging gap: a soname no directory on the path carries. + write_elf_fixture(farm / "libgap.so.1", + {.needed = {"libabsent.so.7"}, .runpath = glibc.string()}); + // The machine's answer: the surface HAS an entry, and it points nowhere. + write_elf_fixture(farm / "libneedsdriver.so.1", + {.needed = {"libdriver.so.1"}, .runpath = glibc.string()}); + std::filesystem::create_symlink("/nonexistent/driver/libdriver.so.1", + farm / "libdriver.so.1"); + + std::vector dirs{farm}; + std::vector search{farm, glibc}; + auto report = elf::inspect_dlopen_surface(dirs, binding_for(payload), search); + + // BOTH DENOMINATORS. A surface that failed to build enumerates nothing and + // every per-member assertion below then passes. + EXPECT_EQ(report.members, 4u); + EXPECT_EQ(report.walked, 3u) << "the dangling entry is not read, and the " + "other three are"; + + ASSERT_EQ(report.findings.size(), 2u); + // Asserted as FIELDS, not as a substring of a message whose wording is + // free to improve. + const elf::DlopenSurfaceFinding* gap = nullptr; + const elf::DlopenSurfaceFinding* machine = nullptr; + for (auto const& f : report.findings) { + if (f.soname == "libabsent.so.7") gap = &f; + if (f.soname == "libdriver.so.1") machine = &f; + } + ASSERT_NE(gap, nullptr); + ASSERT_NE(machine, nullptr); + EXPECT_FALSE(gap->dangling) << "nothing on the surface answers this name"; + EXPECT_EQ(gap->member.filename(), "libgap.so.1"); + EXPECT_TRUE(machine->dangling) + << "the surface carries the entry; the target is the machine's answer"; +} + +// ONE LIBRARY, NOT ITS TWO NAMES. +// +// A farm links `libfoo.so.N` and `libfoo.so.N.M.P` to one file. Walking the +// entries reports every finding twice and calls thirteen libraries twenty-six, +// which is what the first published record did. +TEST(DlopenSurface, CountsALibraryOnceWhenTwoNamesLinkToIt) { + if constexpr (!mcpp::platform::is_linux) + GTEST_SKIP() << "ELF/glibc runtime physics only apply on Linux"; + Tmp t; + auto payload = t.path / "store"; + auto glibc = payload / "2.44" / "lib64"; + auto farm = t.path / "farm"; + std::filesystem::create_directories(glibc); + std::filesystem::create_directories(farm); + write_elf_fixture(glibc / "libc.so.6", {.needed = {}, .runpath = glibc.string()}); + + auto real = t.path / "libtwo.so.1.2.3"; + write_elf_fixture(real, {.needed = {"libabsent.so.7"}, .runpath = glibc.string()}); + std::filesystem::create_symlink(real, farm / "libtwo.so.1.2.3"); + std::filesystem::create_symlink(real, farm / "libtwo.so.1"); + + std::vector dirs{farm}; + std::vector search{farm, glibc}; + auto report = elf::inspect_dlopen_surface(dirs, binding_for(payload), search); + + EXPECT_EQ(report.members, 1u); + EXPECT_EQ(report.walked, 1u); + ASSERT_EQ(report.findings.size(), 1u); + // The SHORTEST name is kept: it is the one a dlopen asks for. + EXPECT_EQ(report.findings.front().member.filename(), "libtwo.so.1"); +} From 75515ca686f2160f1c974866e19eab9b265de912 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Thu, 10 Sep 2026 01:29:06 +0800 Subject: [PATCH 2/4] fix(plan, symbol_provision): a build program's objects stay in this package's images, and vague linkage is not a second provider Two defects that serving the SYCL runtime's OpenCL adapter made active. Both were latent, neither is caused by #596, and both were found by the checks this branch sharpened rather than by reading. A BUILD PROGRAM'S OBJECTS REACHED A DEPENDENCY'S IMAGE. `role = "object"` with no named target attaches to "every linked image", and that read as "every link unit in this plan" -- which includes the shared library a dependency contributes. Measured: `compat:opencl`'s ICD loader, a C library, came out of the link carrying `saxpy_device` and thirty-seven `sycl::` instantiations, 193 dynamic symbols where its own API is 154, and the process held two copies of the device island. `LinkUnit::dependencyOwned` now separates the images this package produces from the ones a dependency contributes. After: 0 sycl symbols, 154 exports, all of them its own `cl*` entry points. Latent until a SYCL project first had a shared dependency, which is what declaring `compat:opencl` did. VAGUE LINKAGE IS NOT A SECOND PROVIDER. `DynamicSymbol` recorded the symbol's TYPE and not its BINDING, so template instantiations, inline functions and vtables -- which the C++ ABI emits into every image and expects the loader to unify -- were counted as a second provider. `hide_static_cxx_runtime` had already written that rule in a comment; nothing enforced it one layer up. The binding is recorded now, weak definitions are counted rather than reported, and the count is printed for the reason every denominator in this area is printed. Both were invisible while the same artifact still had 68 real findings on top of them. One test moved from positional to designated initialisation: adding a field to `Conflict` bound the provider list to a bool -- a string literal converts to one, so it compiled and the list silently became empty. --- ...-09-09-dlopen-surface-and-two-unwinders.md | 30 +++++++++++ CHANGELOG.md | 20 +++++++ src/build/plan.cppm | 14 +++++ src/build/prepare.cppm | 15 +++++- src/build/runtime_validation.cppm | 16 +++++- src/build/symbol_provision.cppm | 27 +++++++++- src/runtime/elf.cppm | 9 ++++ tests/unit/test_symbol_provision.cpp | 53 ++++++++++++++++++- 8 files changed, 178 insertions(+), 6 deletions(-) diff --git a/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md b/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md index f57be4b1..33dc3474 100644 --- a/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md +++ b/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md @@ -616,6 +616,36 @@ predates it, the example's README announces it, and the example runs correctly across it every time nothing throws. #596 is simply the first time something threw. +## 8.5 What serving the OpenCL adapter uncovered + +R7's reversal put a shared library into every SYCL project's plan for the first +time, and two latent defects became active the moment it did. Both are repaired +here; neither is caused by #596, and neither would have been found by reading. + +**A build program's objects reached a dependency's image.** `role = "object"` +with no named target attaches to "every linked image", and that was reading as +"every link unit in this plan" -- which includes the shared library a +dependency contributes. Measured: `compat:opencl`'s ICD loader, a C library, +came out of the link with `saxpy_device` and thirty-seven `sycl::` +instantiations in it, 193 dynamic symbols where its own API is 154, and the +process held two copies of the island. `LinkUnit::dependencyOwned` now says +which images are this package's, and the rule reads "every image THIS PACKAGE +produces". + +**The symbol-provision check could not tell a weak definition from a strong +one.** `DynamicSymbol` recorded the type and not the binding, so vague-linkage +definitions -- template instantiations, inline functions, vtables, which the +C++ ABI emits into every image and expects the loader to unify -- were counted +as a second provider. `hide_static_cxx_runtime`'s own comment had already +written the rule ("must not ... unifying those across the process is the +intended C++ ABI behaviour"); nothing enforced it one layer up. The binding is +recorded now, weak definitions are counted rather than reported, and the count +is printed so "clean" cannot read as "did not look". + +Both were invisible before because the same artifact had 68 real findings +sitting on top of them. A check whose noise is repaired shows what the noise +was covering, which is the third time this issue has produced that shape. + ## 9. What this touches, across the three repositories The review that asks the other question: not "is each repair right" but "what diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a225995..4ef10010 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,26 @@ libgcc_s 本来就在进程里(libstdc++ 需要它),所以这一步只点名一 `12 24 36 48`。**顺带推翻了示例里一条写下来的结论** —— 设备镜像不匹配的那次抛出 并非拦不住,它拦不住只是因为 unwinder 是坏的。 +### 构建程序的对象不再进入依赖的映像 + +`role = "object"` 的 action 在没有点名 target 时按「每一个链接映像」附着,而这句 +一直被读成「计划里的每一个链接单元」—— 其中包含**依赖贡献的共享库**。实测: +`compat:opencl` 的 ICD loader(一个纯 C 库)链完带着 `saxpy_device` 和 37 个 +`sycl::` 实例化,导出 193 个符号而它自己的 API 只有 154 个,进程里于是有两份岛。 +`LinkUnit::dependencyOwned` 现在区分「本包产出的映像」与「依赖贡献的映像」。 + +这条一直潜伏着:直到 SYCL 工程第一次有了共享依赖才被激活。 + +### 重复符号检查按符号绑定分流 + +`DynamicSymbol` 只记了类型不记绑定,于是 vague-linkage 定义(模板实例、inline +函数、vtable —— C++ ABI 要求每个映像各发一份、由加载器统一)被算成了第二个提供者。 +`hide_static_cxx_runtime` 的注释早就写下了这条规则,但上一层没有任何东西执行它。 +现在绑定被记录,weak 定义只计数不算冲突,而且**计数会被打印** —— 否则「干净」读起来 +就等于「没看」。 + +同样是被那 68 条真冲突盖住的:修好噪声之后才看得见噪声盖住了什么。 + ### 重复符号警告会说出 unwinder 的真实后果 该警告此前把后果一律描述为「库自己的那份副本不会被调用」。对 `_Unwind_*` 这一族, diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 1fed0992..34433a1f 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -80,6 +80,19 @@ struct CompileUnit { struct LinkUnit { std::string targetName; enum Kind { Binary, StaticLibrary, SharedLibrary, TestBinary } kind = Binary; + // Does this image belong to a DEPENDENCY rather than to the package being + // built? A `kind = "shared"` dependency contributes a link unit to the + // consumer's plan, and that unit is not one of the consumer's targets. + // + // It decides where a `role = "object"` action's outputs go. "Every linked + // image" is the right default for a build program's objects -- a test + // binary and a static library both need them -- but it was reading as + // "every link unit in the plan", so a device island was linked into a + // dependency's shared library as well. Measured: `compat:opencl`'s ICD + // loader, a C library, came out carrying `saxpy_device` and thirty-seven + // `sycl::` instantiations, and the process then had two copies of the + // island. Latent until a SYCL project first had a shared dependency. + bool dependencyOwned = false; // Normally relative to plan.outputDir. A `role = "object"` action's outputs // land here ABSOLUTE, on purpose: ninja identifies a file by the string an // edge declares, and the action edge declares whatever prepare_actions @@ -1676,6 +1689,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, LinkUnit lu; lu.targetName = dep.target.name; lu.kind = LinkUnit::SharedLibrary; + lu.dependencyOwned = true; lu.output = dep.output; lu.importLibrary = import_library_for(dep.target, naming); if (msvcTarget && !lu.importLibrary.empty()) diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 87e22275..69b9b47f 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -10358,10 +10358,21 @@ prepare_build(bool print_fingerprint, // consumes `lu.objects`, so the objects an action produced // belong there for exactly the reason a compiled `.cpp`'s do: // the target's content is what it was told to contain. - const bool image = lu.kind == mcpp::build::LinkUnit::Binary + // + // AND NOT A DEPENDENCY'S IMAGE. "Every linked image" means + // every image THIS PACKAGE produces; a `kind = "shared"` + // dependency contributes a link unit to this plan and is not + // one of them. Without the qualifier the SYCL example's device + // island was linked into `compat:opencl`'s ICD loader as well + // -- a C library carrying `saxpy_device` -- and the process + // held two copies of it. An action that means to reach a + // dependency's target cannot: it is not this package's to + // fill, and naming it explicitly already fails as unknown. + const bool image = !lu.dependencyOwned + && (lu.kind == mcpp::build::LinkUnit::Binary || lu.kind == mcpp::build::LinkUnit::SharedLibrary || lu.kind == mcpp::build::LinkUnit::StaticLibrary - || lu.kind == mcpp::build::LinkUnit::TestBinary; + || lu.kind == mcpp::build::LinkUnit::TestBinary); const bool wanted = a.targets.empty() ? image : std::find(a.targets.begin(), a.targets.end(), diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index b8562890..411e3977 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -1036,7 +1036,21 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, .defines = names, }); } - report.conflicts = sp::conflicting_exports(*exported, providers); + // WEAK DEFINITIONS ARE COUNTED, NOT REPORTED. + // + // A template instantiation or an inline function is emitted into every + // image that needs it and the loader keeps one; that is the C++ ABI + // working. Reporting it names a correct build. Measured on the SYCL + // example once the real findings were repaired: thirty-nine shared + // symbols remained and thirty-seven were `sycl::queue` and + // `sycl::buffer` instantiations from the same headers libsycl was + // built from -- and the other two were the island's own `extern "C"` + // entry points, which libsycl does not define at all. + auto all = sp::conflicting_exports(*exported, providers); + for (auto const& conflict : all) + if (conflict.isWeak) ++report.sharedWeak; + std::erase_if(all, [](auto const& c) { return c.isWeak; }); + report.conflicts = std::move(all); report.status = report.conflicts.empty() ? sp::Status::Clean : sp::Status::Conflict; findings.push_back({artifact, std::move(report)}); diff --git a/src/build/symbol_provision.cppm b/src/build/symbol_provision.cppm index 7df7d8b7..c5c6bdf3 100644 --- a/src/build/symbol_provision.cppm +++ b/src/build/symbol_provision.cppm @@ -55,6 +55,11 @@ export namespace mcpp::build::symbol_provision { struct Export { std::string name; bool isFunc = false; + // A vague-linkage definition (STB_WEAK): a template instantiation, an + // inline function, a vtable. Every translation unit that needs one emits + // it and the loader unifies them; that is the C++ ABI working, not an + // image displacing a library's own copy. + bool isWeak = false; }; // One object that could also supply a symbol, as the report will name it. @@ -67,6 +72,7 @@ struct Provider { struct Conflict { std::string name; bool isFunc = false; + bool isWeak = false; std::vector alsoProvidedBy; }; @@ -92,6 +98,14 @@ struct Report { std::size_t exported = 0; std::size_t total = 0; std::vector conflicts; + // Shared vague-linkage definitions, counted and not listed. + // + // They are NOT a finding: the C++ ABI emits a template instantiation into + // every image that needs it and expects the loader to keep one. Counting + // them is still worth doing -- a reader who runs `nm -D` sees them and has + // to be told which ones this check decided about, or "clean" reads as + // "did not look". + std::size_t sharedWeak = 0; // Why, for the two non-answers. Empty for Clean and Conflict. std::string reason; @@ -169,7 +183,8 @@ exported_definitions(const mcpp::platform::elf::DynamicSymbols& symbols) { // share an address with relocated data from being excused. if (!symbol.isFunc && symbols.copyRelocations.contains(symbol.value)) continue; - out.push_back(Export{ .name = symbol.name, .isFunc = symbol.isFunc }); + out.push_back(Export{ .name = symbol.name, .isFunc = symbol.isFunc, + .isWeak = symbol.isWeak }); } std::ranges::sort(out, {}, &Export::name); return out; @@ -179,7 +194,8 @@ std::vector conflicting_exports(std::span exports, std::span closure) { std::vector out; for (auto const& exported : exports) { - Conflict conflict{ .name = exported.name, .isFunc = exported.isFunc }; + Conflict conflict{ .name = exported.name, .isFunc = exported.isFunc, + .isWeak = exported.isWeak }; for (auto const& provider : closure) { if (std::ranges::find(provider.defines, exported.name) != provider.defines.end()) @@ -223,6 +239,13 @@ std::string Report::explain(std::string_view artifact) const { body += " Also provided by:\n"; for (auto const& label : providers) body += std::format(" {}\n", label); + if (sharedWeak > 0) + body += std::format( + " ({} vague-linkage definition{} -- template instantiations, inline\n" + " functions, vtables -- {} also shared and are NOT part of this\n" + " finding: the C++ ABI emits one per image and the loader keeps one.)\n", + sharedWeak, sharedWeak == 1 ? "" : "s", + sharedWeak == 1 ? "is" : "are"); // WHY it matters, then what to do — IN THE ORDER THAT ACTUALLY WORKS. // diff --git a/src/runtime/elf.cppm b/src/runtime/elf.cppm index c5fed09d..8deaef93 100644 --- a/src/runtime/elf.cppm +++ b/src/runtime/elf.cppm @@ -137,6 +137,13 @@ struct DynamicSymbol { // dynamic symbol table therefore has exactly one cause — the linker // exported it so that some shared object's reference would bind to it. bool isFunc = false; + // STB_WEAK. A vague-linkage definition -- a template instantiation, an + // inline function, a vtable -- which the C++ ABI emits into every + // translation unit that needs it and expects the loader to unify across + // the process. That is the intended behaviour, not a leak, so a caller + // asking "is this image providing something twice" has to be able to tell + // it from a strong definition that displaces a library's own. + bool isWeak = false; std::uint64_t value = 0; // st_value; the key a copy relocation matches }; @@ -274,6 +281,7 @@ constexpr std::uint64_t kDtGnuHash = 0x6ffffef5; constexpr std::uint64_t kSymEntrySize = 24; constexpr std::uint64_t kRelaEntrySize = 24; +constexpr unsigned char kStbWeak = 2; constexpr unsigned char kSttObject = 1; constexpr unsigned char kSttFunc = 2; constexpr unsigned char kSttGnuIfunc = 10; @@ -731,6 +739,7 @@ inspect_dynamic_symbols(const std::filesystem::path& object) { out.defined.push_back(DynamicSymbol{ .name = std::move(*name), .isFunc = (type == detail::kSttFunc || type == detail::kSttGnuIfunc), + .isWeak = (bind == detail::kStbWeak), .value = *value, }); } diff --git a/tests/unit/test_symbol_provision.cpp b/tests/unit/test_symbol_provision.cpp index b65028f4..eefd60bd 100644 --- a/tests/unit/test_symbol_provision.cpp +++ b/tests/unit/test_symbol_provision.cpp @@ -137,8 +137,14 @@ TEST(SymbolProvision, TheReportNamesEveryProviderAndCapsTheSymbolList) { report.status = sp::Status::Conflict; report.total = 217; for (int i = 0; i < 20; ++i) + // DESIGNATED, not positional. A field added to `Conflict` between + // `isFunc` and `alsoProvidedBy` bound the provider list to a bool + // here -- a string literal converts to one, so it compiled, and the + // provider list silently became empty. report.conflicts.push_back(sp::Conflict{ - std::format("sym{}", i), true, {"/pkg/lib/libz.so.1"}}); + .name = std::format("sym{}", i), + .isFunc = true, + .alsoProvidedBy = {"/pkg/lib/libz.so.1"}}); report.exported = report.conflicts.size(); auto text = report.explain("consumer"); @@ -199,3 +205,48 @@ TEST(SymbolProvision, OrdinaryLinkFlagsDoNotVoidThePredicate) { "-O2", "-Wl,-rpath,$ORIGIN", "-lz", "-Wl,--as-needed", "-Wl,--enable-new-dtags", "-static-libstdc++", "-shared"})); } + +// ── vague linkage is not a second provider ───────────────────────────────── +// +// A template instantiation, an inline function or a vtable is emitted into +// every image that needs it and the loader keeps one. That is the C++ ABI +// working, and reporting it names a correct build. +// +// Measured on the SYCL example once the real findings were repaired: of the +// thirty-nine symbols the image still shared with `libsycl.so.9`, +// thirty-seven were `sycl::queue` and `sycl::buffer` instantiations from the +// same headers libsycl was built from -- and the remaining two were the +// island's own `extern "C"` entry points, which libsycl does not define. A +// check that could not tell binding from name reported all of them. + +TEST(SymbolProvision, AWeakDefinitionIsCarriedThroughAsWeak) { + auto s = image(); + auto weak = func("_ZN4sycl3_V15queueD2Ev"); + weak.isWeak = true; + s.defined.push_back(weak); + s.defined.push_back(func("saxpy_device")); + auto exports = sp::exported_definitions(s); + ASSERT_TRUE(exports.has_value()); + ASSERT_EQ(exports->size(), 2u); + // Sorted by name: the mangled one first. + EXPECT_TRUE((*exports)[0].isWeak); + EXPECT_FALSE((*exports)[1].isWeak); +} + +TEST(SymbolProvision, AConflictRemembersWhetherItsDefinitionIsWeak) { + std::vector exports{ + { .name = "_ZN4sycl3_V15queueD2Ev", .isFunc = true, .isWeak = true }, + { .name = "inflate", .isFunc = true, .isWeak = false }, + }; + std::vector closure{ + { .label = "libsycl.so.9", + .defines = {"_ZN4sycl3_V15queueD2Ev", "inflate"} }, + }; + auto conflicts = sp::conflicting_exports(exports, closure); + ASSERT_EQ(conflicts.size(), 2u); + // Both are shared; only the binding separates them, and the caller is what + // decides which one is a finding. Asserted here rather than in the caller + // so the DATA carries the distinction even if a future caller forgets it. + EXPECT_TRUE(conflicts[0].isWeak); + EXPECT_FALSE(conflicts[1].isWeak); +} From 7d2f77bf889fd7ab6e87d031bd5bd62bf0ac2300 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Thu, 10 Sep 2026 01:32:23 +0800 Subject: [PATCH 3/4] ci: an apt list this job does not use must not fail its setup Two cross-build jobs died in setup with E: Failed to fetch https://dl.google.com/linux/chrome-stable/deb/.../Packages.gz Hash Sum mismatch E: Some index files failed to download. before a single byte was compiled. The runner image carries third-party apt lists that these jobs never install from, and `apt-get update` fails the whole run when any one of them is transiently inconsistent -- so a red that says nothing about the change under test. The lists a job has no use for are removed before the update. What remains is Ubuntu's own, which is what `qemu-user-static`, `wine` and `build-essential` come from. Left alone: the two container-based jobs, whose images carry no third-party lists, and the `|| { ... }` fallback beside the wine dpkg path, which already continues past a failed update. --- .github/workflows/cross-build-test.yml | 16 ++++++++++++++++ .github/workflows/openkal-cross.yml | 8 ++++++++ .github/workflows/release.yml | 8 ++++++++ 3 files changed, 32 insertions(+) diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 9175d63a..78e1f16d 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -99,6 +99,14 @@ jobs: - name: Install qemu-user-static run: | + # The runner image carries third-party apt lists (Google Chrome + # among them) that this job does not use, and a transient + # `Hash Sum mismatch` on one of them fails the whole update -- which + # killed two cross-build jobs in setup, before a single byte was + # compiled. Dropping the lists this job has no use for is what makes + # the step's failure mean something about this job. + sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ + /etc/apt/sources.list.d/microsoft-prod.list || true sudo apt-get update -qq sudo apt-get install -y qemu-user-static ${{ matrix.qemu_bin }} --version | head -1 @@ -251,6 +259,14 @@ jobs: sudo dpkg -i ~/wine-debs/*.deb 2>/dev/null \ || { sudo apt-get update -qq; sudo apt-get install -f -y; } else + # The runner image carries third-party apt lists (Google Chrome + # among them) that this job does not use, and a transient + # `Hash Sum mismatch` on one of them fails the whole update -- which + # killed two cross-build jobs in setup, before a single byte was + # compiled. Dropping the lists this job has no use for is what makes + # the step's failure mean something about this job. + sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ + /etc/apt/sources.list.d/microsoft-prod.list || true sudo apt-get update -qq sudo apt-get install -y --download-only wine64 wine \ || sudo apt-get install -y --download-only wine diff --git a/.github/workflows/openkal-cross.yml b/.github/workflows/openkal-cross.yml index 3ac63d1d..0c7dfb34 100644 --- a/.github/workflows/openkal-cross.yml +++ b/.github/workflows/openkal-cross.yml @@ -368,6 +368,14 @@ jobs: - name: Install the emulators the last two scripts need run: | set -euo pipefail + # The runner image carries third-party apt lists (Google Chrome + # among them) that this job does not use, and a transient + # `Hash Sum mismatch` on one of them fails the whole update -- which + # killed two cross-build jobs in setup, before a single byte was + # compiled. Dropping the lists this job has no use for is what makes + # the step's failure mean something about this job. + sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ + /etc/apt/sources.list.d/microsoft-prod.list || true sudo apt-get update -qq && sudo apt-get install -y -qq qemu-user "$XLINGS_BIN" install xim:qemu-riscv -y XLINGS_HOME="${MCPP_HOME:-$HOME/.mcpp}/registry" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f3f3f0fe..05adbe4d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -275,6 +275,14 @@ jobs: - name: Install system deps + qemu run: | + # The runner image carries third-party apt lists (Google Chrome + # among them) that this job does not use, and a transient + # `Hash Sum mismatch` on one of them fails the whole update -- which + # killed two cross-build jobs in setup, before a single byte was + # compiled. Dropping the lists this job has no use for is what makes + # the step's failure mean something about this job. + sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ + /etc/apt/sources.list.d/microsoft-prod.list || true sudo apt-get update -qq sudo apt-get install -y curl git build-essential qemu-user-static qemu-aarch64-static --version | head -1 From 4efdb4673810ad2a82ee9f99969cced9a142491a Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Thu, 10 Sep 2026 01:34:52 +0800 Subject: [PATCH 4/4] ci: remove the unused apt sources by content, not by filename The first attempt named `google-chrome.list`, and `apt-get update` failed on the same URL: on ubuntu-24.04 the runner writes deb822 `.sources` files, so the filename was a guess and the guess was wrong. Selected by what the file CONTAINS now. That is the same correction `libs/hostlib.lua` records for library directories: a layout you assume is a layout you are wrong about on some machine. --- ...6-09-09-dlopen-surface-and-two-unwinders.md | 3 +++ .github/workflows/cross-build-test.yml | 18 ++++++++++++++---- .github/workflows/openkal-cross.yml | 9 +++++++-- .github/workflows/release.yml | 9 +++++++-- 4 files changed, 31 insertions(+), 8 deletions(-) diff --git a/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md b/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md index 33dc3474..158da2ae 100644 --- a/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md +++ b/.agents/docs/2026-09-09-dlopen-surface-and-two-unwinders.md @@ -662,6 +662,9 @@ else moves when they land". | | a non-hermetic binding, or `allow_host_libs` | silent, for the reason the artifact verdict is | | R6 (one unwinder) | a libc++ link line naming libstdc++ | `--unwindlib=libgcc` and hidden archives | | | every other link | byte-for-byte unchanged, asserted in `test_distribution.cpp` | +| R7 (`compat:opencl` declared) | every SYCL project | the OpenCL back end loads; one shared library and one symlink farm added to the graph | +| | the two defects in §8.5 | latent before, active from the moment a shared library entered a SYCL plan | +| `compat.opencl` on Windows | a Windows OpenCL consumer | a loader to link, where there was none; no adapter, because the system loader needs no help | The one regression this could cause is in §7's last item: an executable that deliberately re-exports the C++ standard library to a plugin it `dlopen`s. It diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 78e1f16d..77ccf587 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -105,8 +105,13 @@ jobs: # killed two cross-build jobs in setup, before a single byte was # compiled. Dropping the lists this job has no use for is what makes # the step's failure mean something about this job. - sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ - /etc/apt/sources.list.d/microsoft-prod.list || true + # + # BY CONTENT, NOT BY FILENAME. The first attempt removed + # `google-chrome.list` and the update failed on the same URL: on + # ubuntu-24.04 the runner writes deb822 `.sources` files, so the + # name was a guess and the guess was wrong. + sudo grep -rlE 'dl[.]google[.]com|packages[.]microsoft[.]com' \ + /etc/apt/sources.list.d/ 2>/dev/null | xargs -r sudo rm -f sudo apt-get update -qq sudo apt-get install -y qemu-user-static ${{ matrix.qemu_bin }} --version | head -1 @@ -265,8 +270,13 @@ jobs: # killed two cross-build jobs in setup, before a single byte was # compiled. Dropping the lists this job has no use for is what makes # the step's failure mean something about this job. - sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ - /etc/apt/sources.list.d/microsoft-prod.list || true + # + # BY CONTENT, NOT BY FILENAME. The first attempt removed + # `google-chrome.list` and the update failed on the same URL: on + # ubuntu-24.04 the runner writes deb822 `.sources` files, so the + # name was a guess and the guess was wrong. + sudo grep -rlE 'dl[.]google[.]com|packages[.]microsoft[.]com' \ + /etc/apt/sources.list.d/ 2>/dev/null | xargs -r sudo rm -f sudo apt-get update -qq sudo apt-get install -y --download-only wine64 wine \ || sudo apt-get install -y --download-only wine diff --git a/.github/workflows/openkal-cross.yml b/.github/workflows/openkal-cross.yml index 0c7dfb34..fe6981b4 100644 --- a/.github/workflows/openkal-cross.yml +++ b/.github/workflows/openkal-cross.yml @@ -374,8 +374,13 @@ jobs: # killed two cross-build jobs in setup, before a single byte was # compiled. Dropping the lists this job has no use for is what makes # the step's failure mean something about this job. - sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ - /etc/apt/sources.list.d/microsoft-prod.list || true + # + # BY CONTENT, NOT BY FILENAME. The first attempt removed + # `google-chrome.list` and the update failed on the same URL: on + # ubuntu-24.04 the runner writes deb822 `.sources` files, so the + # name was a guess and the guess was wrong. + sudo grep -rlE 'dl[.]google[.]com|packages[.]microsoft[.]com' \ + /etc/apt/sources.list.d/ 2>/dev/null | xargs -r sudo rm -f sudo apt-get update -qq && sudo apt-get install -y -qq qemu-user "$XLINGS_BIN" install xim:qemu-riscv -y XLINGS_HOME="${MCPP_HOME:-$HOME/.mcpp}/registry" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 05adbe4d..d8ec4b5e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -281,8 +281,13 @@ jobs: # killed two cross-build jobs in setup, before a single byte was # compiled. Dropping the lists this job has no use for is what makes # the step's failure mean something about this job. - sudo rm -f /etc/apt/sources.list.d/google-chrome.list \ - /etc/apt/sources.list.d/microsoft-prod.list || true + # + # BY CONTENT, NOT BY FILENAME. The first attempt removed + # `google-chrome.list` and the update failed on the same URL: on + # ubuntu-24.04 the runner writes deb822 `.sources` files, so the + # name was a guess and the guess was wrong. + sudo grep -rlE 'dl[.]google[.]com|packages[.]microsoft[.]com' \ + /etc/apt/sources.list.d/ 2>/dev/null | xargs -r sudo rm -f sudo apt-get update -qq sudo apt-get install -y curl git build-essential qemu-user-static qemu-aarch64-static --version | head -1