fix(llvm,llvm20): correct clang's default GCC triple for Azure Linux stage2 by rlmenge · Pull Request #17100 · microsoft/azurelinux

rlmenge · 2026-05-08T22:06:05Z

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

Summary

Fix clang/clang20 stage2 bootstrap failures caused by clang’s generated default GCC triple pointing at the Fedora/RHEL-style runtime path.

Both llvm and llvm20 generate clang config files containing:

--gcc-triple=%{_target_cpu}-redhat-linux

In Azure Linux stage2, GCC runtime files are installed under the Azure Linux vendor triple instead:

/usr/lib/gcc/%{_target_cpu}-%{_vendor}-linux/

This mismatch causes clang-driven links to fail when looking for crtbeginS.o, libgcc, and libgcc_s. That affects both the primary LLVM toolchain and the llvm20 compatibility toolchain during
stage2/self-bootstrap builds, and can also break downstream packages that link through clang.

This change updates both llvm and llvm20 to:

Generate clang/clang20 default configs with %{_target_cpu}-%{_vendor}-linux.
Add a temporary bootstrap clang config during %build so the current self-rebuild overrides the stale clang config already present in the buildroot.
Wrap the instrumented clang used during PGO profile generation so generate-profdata also uses the corrected GCC triple.
Disable %check by default because OpenMP tests are unreliable under systemd-nspawn due to missing /proc/sys/kernel tunables and restricted cgroup/thread support.

The %build bootstrap config injection and PGO clang wrapper are intended as temporary workarounds. The plan is to remove them once stage2 reaches a stable state where a usable clang with the corrected default config is already available in the buildroot.

Change Log

Correct clang and clang20 default GCC triple from *-redhat-linux to *-%{_vendor}-linux.
Add matching llvm and llvm20 bootstrap workarounds using --config=/tmp/azl-clang-bootstrap.cfg.
Apply the bootstrap config to the PGO instrumented clang wrapper for both toolchains.
Disable %check by default for both llvm and llvm20.
Regenerate specs, macro files, and lock fingerprints for both components.

Associated issues

https://dev.azure.com/mariner-org/mariner/_workitems/edit/19791

Test Methodology

Validated both llvm and llvm20 through Koji stage2 builds.

llvm Koji task/build: 2321488
llvm20 Koji task/build: 2321487
Both Koji builds completed successfully and produced the expected clang/LLVM/lld RPMs.

$ clang --version
clang version 21.1.8 (Microsoft Corporation 21.1.8-6.azl4)
Target: x86_64-azurelinux-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Configuration file: /etc/clang/x86_64-azurelinux-linux-gnu-clang.cfg

Additional validation:
azldev comp build -p llvm20
validated against the azl-4.0-stage2-x86_64 mock config and produced clang20/llvm20/lld20 RPMs cleanly.

…stage2 The upstream Fedora clang spec writes a default config drop-in to /etc/clang/<triple>-clang.cfg containing '--gcc-triple=<cpu>-redhat-linux'. Clang autoloads this drop-in when its compiled-in default target matches the basename. In Azure Linux 4.0 stage2, GCC's runtime files (crtbegin.o, crtend.o, libgcc.a, etc.) ship under /usr/lib/gcc/x86_64-azurelinux-linux/ not /usr/lib/gcc/x86_64-redhat-linux/, so clang fails to find crtbeginS.o, -lgcc, and -lgcc_s when invoking ld.lld. This breaks any package whose build links via clang -- most visibly the kernel BPF selftests, which abort with 'cannot open crtbeginS.o' while linking liburandom_read.so, leading to the kernel build aborting on a strict 'cp .../bpftool' for missing artifacts. Add overlays to llvm and llvm20: 1. Flip the cfg_file_content macro from -redhat-linux to -%{_vendor}-linux so the shipped clang RPM's drop-in points at the actual GCC tree. Uses %{_vendor} (evaluates to 'azurelinux' in mock) rather than hardcoding the vendor string. 2. (llvm only) Bootstrap workaround: write a temporary config file with the correct triple and inject '--config=...' into CFLAGS/CXXFLAGS/LDFLAGS, so during the llvm self-rebuild the existing (broken) chroot clang's drop-in is overridden. 3. (llvm only) PGO bootstrap workaround: wrap the just-built instrumented clang in a shell wrapper that adds the same --config flag, so the lit-driven 'generate-profdata' perf-training step picks up the correct triple as well. Mock cannot persist this fix via config_opts['files'] because mockbuild/buildroot.py runs _init_aux_files() before _init_pkg_management() -- the chroot_setup_cmd package install would just overwrite our drop-in. The package-level fix is the only viable persistence path.

Koji stage2 also rebuilds llvm20 against the broken in-chroot clang20 (compat builds pin host_clang_maj_ver=20, so they cannot reuse the fixed llvm/clang21 RPM). Mirror the llvm bootstrap onto llvm20: - Inject --config=/tmp/azl-clang-bootstrap.cfg with the corrected --gcc-triple into CFLAGS/CXXFLAGS/LDFLAGS so the self-rebuild overrides the chroot clang20's stale drop-in. - Wrap the PGO instrumented clang to apply the same config during 'generate-profdata'. - Disable %check by default (build.without = ["check"] + '%bcond_with check') because OpenMP tests have a 95% failure rate under systemd-nspawn (missing /proc/sys/kernel tunables and restricted cgroup access). Matches the equivalent llvm fix. Validated end-to-end with 'azldev comp build -p llvm20' against the azl-4.0-stage2-x86_64 mock config; produces clang20/llvm20/lld20 RPMs cleanly.

Copilot

Pull request overview

This PR updates the LLVM (llvm, llvm20) packaging to fix Azure Linux stage2 bootstrap/self-rebuild failures caused by clang’s default GCC triple pointing at the wrong vendor runtime path.

Changes:

Update clang/clang20 default config generation to use %{_target_cpu}-%{_vendor}-linux instead of *-redhat-linux.
Add stage2 bootstrap workarounds (temporary /tmp clang config injection and a PGO perf-training clang wrapper) to force the corrected GCC triple during the self-rebuild.
Disable %check by default for llvm/llvm20 and regenerate rendered specs/macros + refresh lock fingerprints.

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
specs/l/llvm/llvm.spec	Rendered spec updated: corrected gcc triple in clang config, added bootstrap config injection + PGO wrapper, default `%check` disabled.
specs/l/llvm/llvm.azl.macros	Generated azldev macros updated to include `%_without_check 1`.
specs/l/llvm20/llvm20.spec	Rendered spec updated: adds macros load/source, corrected gcc triple in clang20 config, adds bootstrap config injection + PGO wrapper, default `%check` disabled.
specs/l/llvm20/llvm20.azl.macros	New generated azldev macros file (disables check + mlir by default).
base/comps/llvm/llvm.comp.toml	Component config adds overlays/workarounds for gcc triple + bootstrap/PGO wrapper; disables check/mlir via build.without.
base/comps/llvm20/llvm20.comp.toml	New component config for llvm20 with the same overlays/workarounds; disables check/mlir via build.without.
base/comps/components.toml	Removes inline `[components.llvm20]` entry now that llvm20 has a dedicated `.comp.toml`.
locks/llvm.lock	Updates input fingerprint to match component config changes.
locks/llvm20.lock	Updates input fingerprint to match new component config + overlays.

+echo "--gcc-triple=%{_target_cpu}-%{_vendor}-linux" > /tmp/azl-clang-bootstrap.cfg
+export CFLAGS="${CFLAGS} --config=/tmp/azl-clang-bootstrap.cfg"
+export CXXFLAGS="${CXXFLAGS} --config=/tmp/azl-clang-bootstrap.cfg"
+export LDFLAGS="${LDFLAGS} --config=/tmp/azl-clang-bootstrap.cfg"


+# AZL bootstrap workaround for PGO perf training with the just-built clang
+mv %{builddir_instrumented}/bin/clang %{builddir_instrumented}/bin/clang.real
+cat > %{builddir_instrumented}/bin/clang <<'EOF'
+#!/bin/sh
+exec "$(dirname "$0")/clang.real" --config=/tmp/azl-clang-bootstrap.cfg "$@"
+EOF
+chmod +x %{builddir_instrumented}/bin/clang
 %cmake_build --target generate-profdata
+mv %{builddir_instrumented}/bin/clang.real %{builddir_instrumented}/bin/clang


+echo "--gcc-triple=%{_target_cpu}-%{_vendor}-linux" > /tmp/azl-clang-bootstrap.cfg
+export CFLAGS="${CFLAGS} --config=/tmp/azl-clang-bootstrap.cfg"
+export CXXFLAGS="${CXXFLAGS} --config=/tmp/azl-clang-bootstrap.cfg"
+export LDFLAGS="${LDFLAGS} --config=/tmp/azl-clang-bootstrap.cfg"


+chmod +x %{builddir_instrumented}/bin/clang
 %cmake_build --target generate-profdata
+mv %{builddir_instrumented}/bin/clang.real %{builddir_instrumented}/bin/clang


+description = "Disable %check by default — OpenMP tests fail in nspawn (no /proc/sys/kernel tunables)"
+type = "spec-search-replace"
+regex = '%bcond_without check'
+replacement = '%bcond_with check'
+
+[[components.llvm.overlays]]


+description = "Disable %check by default — OpenMP tests fail in nspawn (no /proc/sys/kernel tunables)"
+type = "spec-search-replace"
+regex = '%bcond_without check'
+replacement = '%bcond_with check'
+
+[[components.llvm20.overlays]]


reubeno · 2026-05-08T22:41:49Z

+build.without = ["mlir", "check"]
+
+[[components.llvm.overlays]]
+description = "Disable %check by default — OpenMP tests fail in nspawn (no /proc/sys/kernel tunables)"


If this is already a bcond in the spec, the .without above should be sufficient.

github-actions · 2026-05-08T22:46:37Z

📄❌ Rendered specs are out of date

FIX: — run this and commit the result:

azldev component render llvm llvm20

Or download the fix patch and apply it:

gh run download 25581854093 -R microsoft/azurelinux -n rendered-specs-patch
git apply rendered-specs.patch

Category	Count
Content diffs	2
Extra files (untracked)	0
Missing files (deleted)	0

Content diffs

`specs/l/llvm/llvm.spec`

--- committed/specs/l/llvm/llvm.spec
+++ rendered/specs/l/llvm/llvm.spec
@@ -2,7 +2,7 @@
 ## (rpmautospec version 0.8.3)
 ## RPMAUTOSPEC: autorelease, autochangelog
 %define autorelease(e:s:pb:n) %{?-p:0.}%{lua:
-    release_number = 6;
+    release_number = 7;
     base_release_number = tonumber(rpm.expand("%{?-b*}%{!?-b:1}"));
     print(release_number + base_release_number - 1);
 }%{?-e:.%{-e*}}%{?-s:.%{-s*}}%{!?-n:%{?dist}}
@@ -3527,6 +3527,9 @@
 
 %changelog
 ## START: Generated by rpmautospec
+* Fri May 08 2026 Rachel Menge <rachelmenge@microsoft.com> - 21.1.8-7
+- fix(llvm): disable checks in nspawn builds
+
 * Fri May 01 2026 Rachel Menge <rachelmenge@microsoft.com> - 21.1.8-6
 - fix(llvm,llvm20): correct clang's default GCC triple for Azure Linux
   stage2

`specs/l/llvm20/llvm20.spec`

--- committed/specs/l/llvm20/llvm20.spec
+++ rendered/specs/l/llvm20/llvm20.spec
@@ -2,7 +2,7 @@
 ## (rpmautospec version 0.8.3)
 ## RPMAUTOSPEC: autorelease, autochangelog
 %define autorelease(e:s:pb:n) %{?-p:0.}%{lua:
-    release_number = 4;
+    release_number = 5;
     base_release_number = tonumber(rpm.expand("%{?-b*}%{!?-b:1}"));
     print(release_number + base_release_number - 1);
 }%{?-e:.%{-e*}}%{?-s:.%{-s*}}%{!?-n:%{?dist}}
@@ -3531,6 +3531,9 @@
 
 %changelog
 ## START: Generated by rpmautospec
+* Fri May 08 2026 Rachel Menge <rachelmenge@microsoft.com> - 20.1.8-5
+- fix(llvm20): bootstrap workaround + disable nspawn-broken checks
+
 * Fri May 01 2026 Rachel Menge <rachelmenge@microsoft.com> - 20.1.8-4
 - fix(llvm,llvm20): correct clang's default GCC triple for Azure Linux
   stage2

rlmenge added 3 commits May 8, 2026 07:05

fix(llvm): disable checks in nspawn builds

6b61bda

rlmenge marked this pull request as ready for review May 8, 2026 22:26

rlmenge requested a review from ddstreetmicrosoft as a code owner May 8, 2026 22:26

Copilot AI review requested due to automatic review settings May 8, 2026 22:26

rlmenge requested review from christopherco and reubeno as code owners May 8, 2026 22:26

Copilot started reviewing on behalf of rlmenge May 8, 2026 22:27 View session

Copilot AI reviewed May 8, 2026

View reviewed changes

rlmenge mentioned this pull request May 8, 2026

fix(llvm): correct clang's default GCC triple for Azure Linux stage2 #17043

Closed

12 tasks

reubeno reviewed May 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(llvm,llvm20): correct clang's default GCC triple for Azure Linux stage2#17100

fix(llvm,llvm20): correct clang's default GCC triple for Azure Linux stage2#17100
rlmenge wants to merge 3 commits intotomls/base/mainfrom
rlmenge/tomls/clang-gcc-triple-bootstrap

rlmenge commented May 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

reubeno May 8, 2026

Uh oh!

github-actions Bot commented May 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

rlmenge commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

reubeno May 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 8, 2026

📄❌ Rendered specs are out of date

Content diffs

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rlmenge commented May 8, 2026 •

edited

Loading