Repository navigation
ci: run the workflow on the local self-hosted runners - #3
Merged
Merged
Conversation
Select the runner fleet through the TRACE_CI_RUNNERS repository variable, like Trace Arena and Forma. Under self-hosted the macOS ARM64 and Linux x64 legs stay; the Windows and Linux arm64 legs are dropped (no Windows machine, and the arm64 Linux container is too small for the native build). Packaging and deploy are skipped for the same reason: they merge the native binaries of every leg, including Windows, and the fork must not publish. The Linux test leg now installs the SDL/X11/GL/Vulkan runtime libraries and starts Xvfb with llvmpipe/lavapipe, because the local runner image is a plain .NET container. The tools tests use $HOME for MGFXC_WINE_PATH instead of the hosted runner's fixed home.
The LunarG installer needs sudo (the runner Mac has no passwordless sudo) and the Wine action runs `brew upgrade`; use the SDK in ~/VulkanSDK and the already-installed Wine/MGFXC prefix instead. Linux installs pkg-config before the Vulkan SDK build, which builds the loader from source.
…ads locally The Wine action shells out to `sudo wget` before it installs anything, and the local runner image has neither wget nor the macOS workload. Install wget/gnupg with pkg-config, and let the macOS workload step run on the self-hosted Mac: MGCB Editor targets the `macos` workload (NETSDK1147). The SDK comes from setup-dotnet in ~/.dotnet, so the install needs no sudo and is a fast no-op after the first job.
The Linux Wine action aborts because it runs apt install without -y on a non-interactive runner; install the same WineHQ build ourselves (without the 2.2 GB of recommends) and let the mgfxc script prepare the prefix. The macOS workload recommends Xcode 26.2 and the runner Mac has 27.1, so accept the host Xcode through ValidateXcodeVersion=false for MGCB Editor and the iOS template. The tests also get a PulseAudio null sink on the Linux container, so the Audio/Song suites run against a real backend instead of a missing device.
zignd
force-pushed
the
ci/self-hosted-runners
branch
from
October 24, 2026 16:56
cc8ba73 to
665af4d
Compare
…load test WineHQ 11 installs a unified wow64 `wine` and no `wine64`, so the guard aborted the build step right after a successful install. The macOS tools suite also exercises the metal cook, which needs Xcode's Metal Toolchain component (installed on the host); the HotReload cancellation test can see one extra compile when macOS replays the watched file's creation event, so it retries.
Cake's GitHub Actions client constructs its own HttpClient, so the HTTPCLIENT_TIMEOUT override in build/Program.cs never reaches the artifact upload and the 100 second default expires on the macOS zips from a home uplink. Under TRACE_CI_RUNNERS=self-hosted the task now stops after the tool store cleanup, and the workflow uploads the three test folders with actions/upload-artifact (chunked, retried). Packaging and Deploy are skipped in that setup, so no other artifact has a consumer.
… off the container
`links {"MoltenVK"}` resolves through the default linker search paths, so a
MoltenVK dylib installed elsewhere on the build machine (a package manager's
/usr/local/lib) was linked instead of the SDK's xcframework archive. The
resulting libmgruntime.dylib kept a @rpath/libMoltenVK.dylib dependency with no
rpath, so the DesktopVK test binaries could not load on any machine without
that dylib. Link the archive by path.
The Linux self-hosted runner is a container with no sound card. A PulseAudio
null sink makes SDL open a device, but every stream's first callback arrives
~2 s late, so the playback-timing assertions cannot hold; upstream already
gates these suites on having an audio device. Keep them on the macOS leg
(real CoreAudio) and off the Linux container.
`links` prefixes library names with -l, which turns the archive path into `-llibMoltenVK.a`; linkoptions emits the absolute archive path as-is.
The per-fixture Graphics suites render and compare reference images, so they need a real device: on the Linux self-hosted container (Mesa llvmpipe) the DesktopGL suites die with SIGSEGV as soon as they open a device, while the non-graphical suites (Framework, Input, Visual, Game) pass. Gate those steps on a job-level GPU_SUITES flag that only the macOS leg sets. Two macOS image comparisons fail on DesktopGL as well as on DesktopVK, where they were already ignored: the reference images are Windows captures and neither MoltenVK nor Apple's OpenGL reproduces the address-mode wrapping or the mip chain. Extend the existing macOS ignore to every backend.
The Vulkan assemblies cannot work on our legs: the Linux container fails 70 of 590 tests (SDL3 window/input smoke tests on headless Xvfb + lavapipe), and the Mac fails 5 (the DesktopVK HiDPI patch makes the back buffer the drawable size, so GraphicsDeviceManagerTest expects 800 and gets 1600 on Retina, and SDL3 refuses a window from the test thread in GameStepFrameTest). The build leg still compiles the Vulkan runtime and its shaders, which is the part that can regress on these machines.
The suite builds a Game on the NUnit worker thread, so SDL2's Cocoa backend registers the app and sets the main menu from that thread and the process aborts with NSInternalInconsistencyException (exit 134). It passed once by luck when the main thread initialized video first, so it is the fixture, not the runner. The Linux leg keeps running the suite.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Selects the runner fleet through
TRACE_CI_RUNNERS(same flag as Trace Arena and Forma). Unset keeps GitHub-hosted runners;self-hosteduses the local fleet. Green run with the flag: https://github.com/zigrok/MonoGame/actions/runs/40127039576 (build-macos,build-linux,tests-macos,tests-linux).Under
self-hosted:buildandtestskeep the macOS ARM64 and Linux x64 legs. Thewindowsandlinux-armlegs are dropped (no Windows machine here; the arm64 Linux container is 1.5 GB). Hosted CI keeps all four legs.PackNativeRuntime,PackageBinariesandDeployare skipped (they merge native binaries from every leg including Windows, and the fork must not publish).testsno longer needspack-native-runtime, which would otherwise skip the whole test matrix when packaging is skipped.actions/upload-artifactinstead of the build script'sUploadArtifacts. That task goes through Cake's GitHub Actions client, which hardcodesnew HttpClient()— the 100 s default timeout thatHTTPCLIENT_TIMEOUTcannot override — and the macOS zips exceed it from a home uplink. Only the test folders are uploaded, which is all the tests job consumes once packaging is skipped.pkg-config,wget,gnupg) and install Wine from WineHQ themselves: the upstream action runsapt installwithout-yand aborts on a non-interactive runner. WineHQ 11 ships the unified wow64wineonly, so the guard checkswine, notwine64.Xvfb(llvmpipe/lavapipe) and setsXDG_RUNTIME_DIR, which SDL needs.~/VulkanSDKdirectory that actually hasinclude/andlib/), the host's Wine, andValidateXcodeVersion=false(the workload recommends Xcode 26.2, the host has 27.1). Themacos/iosworkloads are installed into~/.dotnet, which setup-dotnet owns, so no sudo is needed and later jobs reuse them.Suites that need hardware the local legs do not have are gated the way upstream already gates them (
runner.environment != 'github-hosted'), with the measured reason in a comment:Graphics.*image-comparison suites on Linux (SIGSEGV on Mesa llvmpipe as soon as a device is opened). They run on the macOS leg.Gamesuite on macOS: the fixture builds aGame— and so initializes SDL video — on the NUnit worker thread, and SDL2's Cocoa backend then sets the main menu from that thread, aborting withNSInternalInconsistencyException. It runs on the Linux leg.GraphicsDevicepass (references are DirectX/OpenGL captures MoltenVK does not reproduce) and 5 in the non-visual pass (the DesktopVK HiDPI patch makes the back buffer the drawable size; SDL3 refuses a window from the test thread). The build leg still builds the Vulkan runtime, shaders and tools tests.Two defects fixed along the way, both independent of the runners:
-lMoltenVK, so a MoltenVK dylib installed elsewhere on the build host won and the runtime kept an@rpath/libMoltenVK.dylibdependency with no rpath; the Vulkan test binaries could not load it. It now links the SDK'slibMoltenVK.aby path.SamplerStateTest.VisualTestAddressModesandRenderTarget2DTest.GenerateMipsfailed their reference comparison on Apple's OpenGL as well as on MoltenVK (the references are Windows captures); the macOS ignore that existed for Vulkan now covers every backend.SupersededCompileIsCancelledgets[Retry(3)](macOS can replay the watcher's file-creation event and start a third compile on a loaded runner).Host prerequisites for the macOS leg: Rosetta 2 (
softwareupdate --install-rosetta --agree-to-license), the Vulkan SDK unpacked under~/VulkanSDK, Wine plus the~/.winemonogameprefix, and the Metal toolchain component (xcodebuild -downloadComponent MetalToolchain), without which the content pipeline cannot cook effects.