macos: bundle Homebrew dylibs into the .pkg framework; cover arm64 .pkg in test-distribution by Fedr · Pull Request #6098 · MeshInspector/MeshLib

Fedr · 2026-05-14T13:38:23Z

Problem

The macOS .pkg shipped only the MeshLib binaries and offloaded every runtime dep to Homebrew via requirements/macos.txt. When a Homebrew bottle drops a SOMAJOR symlink (e.g. jsoncpp 1.9.6 no longer ships libjsoncpp.27.dylib), every binary in an already-published .pkg fails to launch with a dyld: Library not loaded abort even though brew install succeeds. The triggering breakage on master: run 25858581152, job 75992495208.

dyld: Library not loaded: /usr/local/opt/jsoncpp/lib/libjsoncpp.27.dylib
  Referenced from: /Library/Frameworks/MeshLib.framework/.../bin/MeshViewer
  Reason: tried: '/usr/local/opt/jsoncpp/lib/libjsoncpp.27.dylib' (no such file), ...

Fix

Bundle Homebrew dylib deps into the framework so the .pkg is self-contained for the fragile bits. The .pkg no longer relies on any Homebrew symlink shape — only on Homebrew being present for libpython3.10 and the toolchain.

`scripts/macos_bundle_dylibs.py` (new)

Walks every Mach-O under MeshLib.framework/Versions/<X.Y.Z.W>/{bin,lib} and, for each load command:

Absolute Homebrew path (/usr/local/..., /opt/homebrew/..., or whatever brew --prefix reports) — copy the resolved real file into .../lib/ and rewrite the LC_LOAD_DYLIB to @rpath/<basename>.
@rpath/<X> / @loader_path/<X> — modern Homebrew bottles install with @rpath/<basename>, so the load command never mentions /usr/local/.... If X isn't a MeshLib-shipped lib, resolve it via:
1. The original source directory of the bundled lib that referenced it (handles intra-bottle siblings like gdcm's internal libsocketxx.1.2.dylib).
2. Fallback: any Homebrew lib dir (lib, opt/*/lib, Cellar/*/*/lib).
System libs (/usr/lib, /System) and libpython* / Python framework — left as external references so the host system's Python is used.

The dst basename is the referrer's requested name, not the realpath's name — so libglfw.3.4.dylib (the realpath after symlink resolution) is bundled as libglfw.3.dylib (what the binary asks for via @rpath/libglfw.3.dylib). Same shape for fmt / tbb / spdlog / openvdb / opencascade etc. Otherwise dyld would not find the bundled copy by the name it actually asks for.

After copying, each Mach-O is rewritten via install_name_tool (id, load commands), an rpath is added (@executable_path/../lib for executables, @loader_path/. for dylibs), and the file is ad-hoc re-signed via codesign -s -.

Wired into scripts/distribution_apple.sh right after cmake --install.

`test-distribution.yml`

Adds a macos-arm64-test job mirroring macos-x64-test on macos-latest (github-hosted Apple Silicon) pulling *arm.pkg. Previously only the x64 .pkg was end-to-end-tested, leaving arm64 packaging regressions invisible.

Why this is robust to future Homebrew drift

Once a .pkg is built, the binaries inside reference only @rpath/<basename> for their bundled deps; dyld resolves that to the copy under Versions/Current/lib/, which is part of the .pkg payload. Future SOMAJOR bumps or alias removals in Homebrew cannot break already-installed binaries.

requirements/macos.txt and the xargs brew install step in test-distribution.yml are kept untouched on purpose — they're harmless once load commands are @rpath/... (bundled libs win via the new rpath), and keeping them avoids churn in the dev install / NuGet patch paths.

Validated in CI

Latest run 25916902755 — all green including both new end-to-end tests:

macos-build-test (x64, Release) ✅
macos-build-test (arm64, Release) ✅
macos-build-test (arm64, Debug) ✅
test-distribution / macos-x64-test ✅ — installs meshlib_*x64.pkg, launches MeshViewer from the installed framework
test-distribution / macos-arm64-test ✅ — same for meshlib_*arm.pkg on Apple Silicon

.pkg size impact

arch	master	PR
x64	40.4 MB	81.7 MB
arm	39.1 MB	77.1 MB

The ~40 MB delta on each arch is the bundled Homebrew runtime tree (boost, opencascade libTK*, openvdb, tbb, fmt, spdlog, jsoncpp, glfw, libpng, libtiff, libharu, etc.) — payload that previously the user had to install via brew install and that broke whenever a bottle's SOMAJOR symlink changed.

Iteration log (debugging the bundling design in CI)

For future spelunkers, each green push uncovered the next hidden assumption in the original design:

First pass only chased absolute Homebrew paths → bundled gdcm but missed internal siblings (@rpath/libsocketxx.1.2.dylib from libgdcmMEXD).
Second pass added intra-bottle @rpath/X resolution via source-dir lookup → bundled libsocketxx, but MeshViewer still failed on @rpath/libglfw.3.dylib.
Third pass treated @rpath/X as bundleable even on MeshLib's own binaries (since modern bottles install with @rpath/<basename> install_name, so MeshLib binaries store @rpath/... too) — but bundled libglfw.3.4.dylib (the realpath) instead of libglfw.3.dylib (the requested basename).
Fourth pass preserves the referrer's requested basename. x64 test passed. arm64 silently bundled 0 dylibs because the self-hosted arm runner has Homebrew at /Users/runner/.homebrew, not /usr/local/ or /opt/homebrew/.
Fifth pass detects the Homebrew prefix at startup via brew --prefix. Both archs now bundle ~40 MB of dylibs and pass the full end-to-end test.

The macOS distribution leaves runtime deps to Homebrew via requirements/macos.txt, so when a bottle's SOMAJOR alias disappears (e.g. jsoncpp dropping libjsoncpp.27.dylib) every installed MeshLib binary fails to launch with a dyld "Library not loaded" error even though `brew install` succeeds. Add scripts/macos_bundle_dylibs.py to walk Mach-O files under the framework's bin/ and lib/, copy any Homebrew dep into lib/, rewrite the load commands to @rpath/<basename>, add an @executable_path/../lib (for executables) or @loader_path/. (for dylibs) rpath, and ad-hoc re-sign. System libs and libpython* are left as external references on purpose. Invoke it from scripts/distribution_apple.sh after `cmake --install`.

Bottles often reference internal siblings via @rpath/<name> (e.g. gdcm's libgdcmMEXD references @rpath/libsocketxx.1.2.dylib at @loader_path/.) rather than absolute paths. Initial bundling logic only chased absolute Homebrew paths and stopped at @rpath/..., leaving siblings unbundled and MeshViewer aborting on a fresh install with "Library not loaded: @rpath/libsocketxx.1.2.dylib". Track each bundled lib's original source directory and resolve any @rpath/X or @loader_path/X dep encountered inside that lib against the source dir first, then fall back to the standard Homebrew lib dir for cross-package references.

Modern Homebrew bottles install with LC_ID_DYLIB = @rpath/<basename>, so MeshLib binaries linked against them store the dep as @rpath/<basename> rather than an absolute /usr/local/... path. Previous bundling logic only chased absolute Homebrew paths, leaving libs like glfw (referenced as @rpath/libglfw.3.dylib by MeshViewer) unbundled. The installed .pkg then aborted with "Library not loaded: @rpath/libglfw.3.dylib" because none of the CMake-set rpaths resolve to Homebrew's location. Extend the BFS: for any @rpath/X or @loader_path/X dep whose basename isn't already shipped in the framework's lib tree (own libs + thirdparty libs copied in by distribution_apple.sh), look X up under any Homebrew lib dir (/usr/local/lib, /usr/local/opt/*/lib, /usr/local/Cellar/*/*/lib and the /opt/homebrew/ equivalents) and bundle it. Cache the lookup.

Homebrew bottles install both a version-tagged real file (libglfw.3.4.dylib) and SOMAJOR symlinks (libglfw.3.dylib) that point at it. Binaries link via the SOMAJOR symlink, so their LC_LOAD_DYLIB stores "@rpath/libglfw.3.dylib". Previous bundling logic resolved the symlink chain, copied the file under the realpath's basename (libglfw.3.4.dylib), and left the binary still asking dyld for libglfw.3.dylib -- which is no longer present anywhere on a fresh test machine. Same shape for fmt, tbb, spdlog, openvdb, opencascade, etc. Plumb the referrer's requested basename through bundle_from so the dst inside the framework lib/ matches the name the load command actually references. install_name_tool -id (run later in the rewrite pass) then stamps the bundled file's own id with that same name.

Hardcoded HOMEBREW_PREFIXES only covered /usr/local and /opt/homebrew, but the arm64 Release build runs on a self-hosted macos runner whose Homebrew lives at /Users/runner/.homebrew. There the script silently bundled 0 dylibs (arm .pkg size unchanged at 39 MB while x64 doubled to 82 MB), leaving the arm64 .pkg as fragile to bottle drift as before. Call `brew --prefix` at startup and prepend the result to the prefix tuple. Falls back to the defaults if brew is not on PATH (e.g. for local syntax checks on non-mac dev machines).

test-distribution validated only the x64 macOS .pkg, so the arm64 .pkg was uploaded to releases without any end-to-end install/launch check -- mirroring the gap that linux-x64-test / linux-arm64-test long ago closed for Linux. Split the single macos-test job into macos-x64-test (existing, macos-15-intel, *x64.pkg) and a new macos-arm64-test (macos-latest github-hosted Apple Silicon, *arm.pkg). Steps are otherwise identical: install via sudo installer, brew install runtime deps from the bundled requirements/macos.txt, run MeshViewer / meshconv, build C and C++ examples against the installed framework.

…bundle-dylibs

This reverts commit f07dde1.

oitel · 2026-05-15T13:55:07Z

          cmake --build build
-
+
+  macos-arm64-test:


Can it be generalized using a matrix instead?

oitel · 2026-05-15T13:56:33Z

+    return sorted(p for p in root.rglob("*") if is_macho(p))
+
+
+def otool_L(p: Path) -> list[str]:


Give it a reasonable name.

oitel · 2026-05-15T13:58:25Z

Could this self-written script be replaced with an already existing external utility like dylibbundler or by extending the CMake config?

oitel · 2026-05-15T14:02:17Z

+                    break
+            if result is not None:
+                break
+        if result is not None:
+            break


Why not early exit?

@rpath

- Collapse the macos-x64-test / macos-arm64-test pair in test-distribution.yml into a single matrixed macos-test job. Steps were identical other than runner and *.pkg pattern. - Rename otool_L to get_load_dylibs; it's a wrapper that lists the load-command dylibs and the new name says so. - Drop the nested break-and-check pattern in _resolve_homebrew_basename and just return early on first hit. - Note in the script docstring why we didn't lean on dylibbundler or CMake BundleUtilities (referrer-basename preservation, runtime Homebrew-prefix detection for the arm64 self-hosted runner, intra-bottle @rpath sibling resolution, no extra build-time dependency).

Fedr added disable-build-windows disable-build-ubuntu-x64 disable-build-ubuntu-arm64 disable-build-emscripten disable-build-linux-vcpkg upload-binaries Upload built installers and packages to the Releases page labels May 14, 2026

Fedr and others added 7 commits May 14, 2026 20:22

Merge remote-tracking branch 'origin/ci/macos-test-arm64' into macos/…

c7533af

…bundle-dylibs

Switch to a different way of linking LLVM in mrbind.

f07dde1

Fedr mentioned this pull request May 15, 2026

ci: cover both x64 and arm64 macOS .pkg in test-distribution #6103

Closed

2 tasks

Fedr changed the title ~~macos: bundle Homebrew dylibs into the .pkg framework~~ macos: bundle Homebrew dylibs into the .pkg framework; cover arm64 .pkg in test-distribution May 15, 2026

Fedr added 2 commits May 15, 2026 16:50

Revert "Switch to a different way of linking LLVM in mrbind."

eb852f2

This reverts commit f07dde1.

Merge branch 'master' into macos/bundle-dylibs

90fc54d

Fedr requested a review from oitel May 15, 2026 13:52

oitel reviewed May 15, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macos: bundle Homebrew dylibs into the .pkg framework; cover arm64 .pkg in test-distribution#6098

macos: bundle Homebrew dylibs into the .pkg framework; cover arm64 .pkg in test-distribution#6098
Fedr wants to merge 11 commits into
masterfrom
macos/bundle-dylibs

Fedr commented May 14, 2026 •

edited

Loading

Uh oh!

oitel May 15, 2026

Uh oh!

oitel May 15, 2026

Uh oh!

oitel May 15, 2026

Uh oh!

oitel May 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		return sorted(p for p in root.rglob("*") if is_macho(p))


		def otool_L(p: Path) -> list[str]:

Conversation

Fedr commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

scripts/macos_bundle_dylibs.py (new)

test-distribution.yml

Why this is robust to future Homebrew drift

Validated in CI

.pkg size impact

Iteration log (debugging the bundling design in CI)

Uh oh!

oitel May 15, 2026

Choose a reason for hiding this comment

Uh oh!

oitel May 15, 2026

Choose a reason for hiding this comment

Uh oh!

oitel May 15, 2026

Choose a reason for hiding this comment

Uh oh!

oitel May 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fedr commented May 14, 2026 •

edited

Loading

`scripts/macos_bundle_dylibs.py` (new)

`test-distribution.yml`