Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
5278297
Use cargo patch for host as well. Benchmark patched vs unpatched
IvanIsCoding Sep 7, 2026
aa26be6
Add extra logs to cargo patch
IvanIsCoding Sep 7, 2026
d86bad9
Fix for cargo-reproducibility
IvanIsCoding Sep 7, 2026
8e3e8b6
Towards a reproducible build
IvanIsCoding Sep 7, 2026
e0466b2
Try to reproduce linux musl builds
IvanIsCoding Sep 7, 2026
cb87d14
Update reproducibility patch
IvanIsCoding Sep 7, 2026
ed3e5f2
Add entries for aarch64 musl builds
IvanIsCoding Sep 7, 2026
bb193f3
Use aarch64 to verify x86-64 and freebsd x86-64 to verify aarch64
IvanIsCoding Sep 8, 2026
1805b15
Use cargo-auditable for reproducible builds again
IvanIsCoding Sep 8, 2026
f4ff7a2
Use cargo-auditable for reproducible builds again (pt 2)
IvanIsCoding Sep 8, 2026
edecd64
Re-enable mimalloc for reproducible build
IvanIsCoding Sep 8, 2026
4413d37
Merge remote-tracking branch 'origin/main' into reproducibility
IvanIsCoding Sep 8, 2026
bf6aed7
Add xwin to reproducible build setup
IvanIsCoding Sep 8, 2026
05b2458
Use cache for xwin reproducible build
IvanIsCoding Sep 8, 2026
eb7bfdb
Use xwin from windows arm64 for reproducibility check
IvanIsCoding Sep 8, 2026
35680b0
Verbose mode for cargo-xwin
IvanIsCoding Sep 8, 2026
2b9547b
Separate manual xwin builds for now
IvanIsCoding Sep 8, 2026
0735d52
Use patched cargo for Windows arm64 builds
IvanIsCoding Sep 8, 2026
23b391f
let me trigger the workflow
IvanIsCoding Sep 8, 2026
e04bb91
Fix cargo patching
IvanIsCoding Sep 8, 2026
468cf17
Another custom cargo patch
IvanIsCoding Sep 8, 2026
4dbc58f
pdb for debugging
IvanIsCoding Sep 8, 2026
2a6020b
Do not generate PDB for Windows build
IvanIsCoding Sep 8, 2026
2683ed2
Force windows arm64 cargo xwin
IvanIsCoding Sep 8, 2026
9b02796
Upload zip as well
IvanIsCoding Sep 8, 2026
c6cdce6
Reproducible 7z for zips
IvanIsCoding Sep 8, 2026
8e5a2ec
Tidy up
IvanIsCoding Sep 8, 2026
f6b093c
Merge remote-tracking branch 'origin/main' into reproducibility
IvanIsCoding Sep 8, 2026
4703d28
macOS reproducible build attempt #1
IvanIsCoding Sep 8, 2026
0ce3574
Verbose macOS build
IvanIsCoding Sep 9, 2026
6861fcb
Patch cargo for macOS as well
IvanIsCoding Sep 9, 2026
752af97
Use all-static feature from cargo
IvanIsCoding Sep 9, 2026
b399093
Fix miscompilation for macOS intel
IvanIsCoding Sep 9, 2026
8b100b0
Update patchh
IvanIsCoding Sep 9, 2026
21ad83a
Use reproducible gtar command
IvanIsCoding Sep 9, 2026
859b96e
macOS reproducible builds
IvanIsCoding Sep 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/reproducibility/patches/cargo-macos-aarch64-metadata.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
diff --git a/src/cargo/core/compiler/build_runner/compilation_files.rs b/src/cargo/core/compiler/build_runner/compilation_files.rs
--- a/src/cargo/core/compiler/build_runner/compilation_files.rs
+++ b/src/cargo/core/compiler/build_runner/compilation_files.rs
@@ -864,6 +864,12 @@
for line in bcx.rustc().verbose_version.lines() {
if unit.kind.is_host() || !line.starts_with("host: ") {
+ // Normalize hash input only; keep the real host for compilation.
+ let line = if line.starts_with("host: ") {
+ "host: aarch64-apple-darwin"
+ } else {
+ line
+ };
line.hash(hasher);
}
}
return;
@@ -881,3 +887,3 @@
if unit.kind.is_host() {
- bcx.rustc().host.hash(hasher);
+ "aarch64-apple-darwin".hash(hasher);
}
9 changes: 9 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,15 @@ jobs:
- name: Restore Cargo downloads
if: runner.os == 'Linux'
uses: $/.github/actions/restore-cargo-downloads

- name: Select pinned macOS SDK
if: runner.os == 'macOS'
env:
DEVELOPER_DIR: /Applications/Xcode_16.4.app/Contents/Developer
run: |
echo "DEVELOPER_DIR=${DEVELOPER_DIR}" >> "$GITHUB_ENV"
echo "MACOSX_DEPLOYMENT_TARGET=11.0" >> "$GITHUB_ENV"
echo "SDKROOT=$(xcrun --sdk macosx15.5 --show-sdk-path)" >> "$GITHUB_ENV"

- name: Install Rust ${{ matrix.rust }}
uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
Expand Down
183 changes: 181 additions & 2 deletions .github/workflows/manual_reproducible.yml
Original file line number Diff line number Diff line change
Expand Up @@ -490,8 +490,7 @@ jobs:
XWIN_SDK_VERSION: ${{ steps.setup-xwin.outputs.xwin_sdk_version }}
XWIN_VERSION: ${{ steps.setup-xwin.outputs.xwin_version }}
run: >-
cargo xwin build --locked --release --target "${TARGET}"
--no-default-features --features from-toml,from-yaml,greppable,from-xml
cargo auditable xwin build --locked --release --target "${TARGET}"
--xwin-version "${XWIN_VERSION}"
--xwin-sdk-version "${XWIN_SDK_VERSION}"
--xwin-crt-version "${XWIN_CRT_VERSION}"
Expand Down Expand Up @@ -549,3 +548,183 @@ jobs:
(cd artifacts/windows-2025 && sha256sum celq.exe celq-windows-x86_64.zip) > original.sha256
(cd artifacts/windows-11-vs2026-arm && sha256sum celq.exe celq-windows-x86_64.zip) > verification.sha256
diff -u original.sha256 verification.sha256

patched_cargo_mac:
name: Patched Cargo (x86_64-apple-darwin)
runs-on: macos-15
defaults:
run:
shell: bash
env:
DEVELOPER_DIR: /Applications/Xcode_16.4.app/Contents/Developer
MACOSX_DEPLOYMENT_TARGET: "11.0"
TARGET: x86_64-apple-darwin
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false

- name: Checkout Cargo source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
repository: rust-lang/cargo
ref: 29ea6fb6a5db279426f4cc4e17aa385f05a0cfbc
path: cargo-source
persist-credentials: false
fetch-depth: 1
sparse-checkout: |
src
crates
credential
benches/benchsuite
benches/capture

- name: Select pinned macOS SDK
run: echo "SDKROOT=$(xcrun --sdk macosx15.5 --show-sdk-path)" >> "$GITHUB_ENV"

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@9376cdc5a5e25b16da71af47712785cf06b0d6d4 # 1.94.1
with:
targets: ${{ env.TARGET }}

- name: Patch Cargo
working-directory: cargo-source
run: |
git apply --check ../.github/reproducibility/patches/cargo-macos-aarch64-metadata.patch
git apply ../.github/reproducibility/patches/cargo-macos-aarch64-metadata.patch

- name: Cross-compile patched Cargo
working-directory: cargo-source
env:
CFG_RELEASE: 1.94.1
CFG_RELEASE_CHANNEL: stable
CFG_VER_DESCRIPTION: macos-aarch64-metadata-patch
run: cargo build --locked --release --package cargo --bin cargo --features all-static --target "$TARGET"

- name: Upload patched Cargo
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: patched-cargo-x86_64-apple-darwin
path: cargo-source/target/x86_64-apple-darwin/release/cargo
retention-days: 1
if-no-files-found: error

macos:
name: macOS (${{ matrix.runner }}, ${{ matrix.target }})
needs: patched_cargo_mac
runs-on: ${{ matrix.runner }}
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
runner: [macos-15, macos-15-intel]
target: [aarch64-apple-darwin]
env:
DEVELOPER_DIR: /Applications/Xcode_16.4.app/Contents/Developer
MACOSX_DEPLOYMENT_TARGET: "11.0"
NAME: celq
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false

- name: Select pinned macOS SDK
run: |
xcodebuild -version
echo "SDKROOT=$(xcrun --sdk macosx15.5 --show-sdk-path)" >> "$GITHUB_ENV"

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@9376cdc5a5e25b16da71af47712785cf06b0d6d4 # 1.94.1
with:
targets: ${{ matrix.target }}

- name: Install cargo-auditable
uses: taiki-e/install-action@1ed6d7be6168f6c9046541087ff549b6bc581fdf # v2.87.2
with:
tool: cargo-auditable@0.7.5
fallback: none

- name: Download patched Cargo
if: matrix.runner == 'macos-15-intel'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: patched-cargo-x86_64-apple-darwin
path: patched-cargo

- name: Select patched Cargo
if: matrix.runner == 'macos-15-intel'
run: |
chmod +x "$GITHUB_WORKSPACE/patched-cargo/cargo"
echo "$GITHUB_WORKSPACE/patched-cargo" >> "$GITHUB_PATH"
echo "CARGO=$GITHUB_WORKSPACE/patched-cargo/cargo" >> "$GITHUB_ENV"
echo "RUSTC=$(rustup which rustc)" >> "$GITHUB_ENV"
echo "RUSTDOC=$(rustup which rustdoc)" >> "$GITHUB_ENV"

- name: Configure reproducible builds
run: |
echo "SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)" >> "$GITHUB_ENV"
echo "TZ=UTC" >> "$GITHUB_ENV"
echo "LC_ALL=C" >> "$GITHUB_ENV"
RUST_SYSROOT=$(rustc --print sysroot)
echo "RUSTFLAGS=--remap-path-prefix=${CARGO_HOME:-$HOME/.cargo}=/cargo --remap-path-prefix=${RUST_SYSROOT}=/rust-toolchain --remap-path-prefix=${GITHUB_WORKSPACE}=/src" >> "$GITHUB_ENV"

- name: Build
run: cargo auditable build --locked --release --target ${{ matrix.target }}

- name: Install GNU tar
run: brew install gnu-tar

- name: Create release archive
env:
TARGET: ${{ matrix.target }}
PRETTY_TARGET: macos-aarch64
run: |
BIN_PATH="target/${TARGET}/release/${NAME}"
ARCHIVE_NAME="${NAME}-${PRETTY_TARGET}.tar.gz"
gtar --format=gnu --sort=name --mtime="@${SOURCE_DATE_EPOCH}" \
--owner=0 --group=0 --numeric-owner --mode=0755 \
-cf "${ARCHIVE_NAME%.gz}" -C "target/${TARGET}/release" "${NAME}"
gzip -n -9 "${ARCHIVE_NAME%.gz}"

mkdir -p artifacts
cp "${BIN_PATH}" "${ARCHIVE_NAME}" artifacts/

- name: Upload archive and binary
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: reproducible-aarch64-apple-darwin-${{ matrix.runner }}
path: |
artifacts/celq
artifacts/celq-macos-aarch64.tar.gz
retention-days: 1
if-no-files-found: error
compression-level: 0

macos_hashes:
name: SHA-256 (macOS ARM64 vs Intel hosts)
needs: macos
if: ${{ !cancelled() }}
runs-on: ubuntu-24.04
steps:
- name: Download ARM64-host artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: reproducible-aarch64-apple-darwin-macos-15
path: artifacts/macos-15

- name: Download Intel-host artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: reproducible-aarch64-apple-darwin-macos-15-intel
path: artifacts/macos-15-intel

- name: Print and compare SHA-256 hashes
run: |
find artifacts -type f -exec sha256sum {} +
(cd artifacts/macos-15 && sha256sum celq celq-macos-aarch64.tar.gz) > original.sha256
(cd artifacts/macos-15-intel && sha256sum celq celq-macos-aarch64.tar.gz) > verification.sha256
diff -u original.sha256 verification.sha256
20 changes: 16 additions & 4 deletions .github/workflows/release_github.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ jobs:
name: ${{ matrix.target }}
runs-on: ${{ matrix.os }}
environment: github_release
env:
DEVELOPER_DIR: /Applications/Xcode_16.4.app/Contents/Developer
MACOSX_DEPLOYMENT_TARGET: "11.0"
permissions:
contents: write
id-token: write
Expand All @@ -47,6 +50,9 @@ jobs:
with:
persist-credentials: false

- name: Select pinned macOS SDK
run: echo "SDKROOT=$(xcrun --sdk macosx15.5 --show-sdk-path)" >> "$GITHUB_ENV"

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@9376cdc5a5e25b16da71af47712785cf06b0d6d4 # 1.94.1
with:
Expand All @@ -69,15 +75,21 @@ jobs:
- name: Build
run: cargo auditable build --locked --release --target ${{ matrix.target }}

- name: Install GNU tar
run: brew install gnu-tar

- name: Create release archive
id: archive
run: |
BIN_PATH="target/${{ matrix.target }}/release/${NAME}"
ARCHIVE_NAME="${NAME}-${{ matrix.pretty_target }}.tar.gz"
tar czf "${ARCHIVE_NAME}" -C "target/${{ matrix.target }}/release" "${NAME}"
gtar --format=gnu --sort=name --mtime="@${SOURCE_DATE_EPOCH}" \
--owner=0 --group=0 --numeric-owner --mode=0755 \
-cf "${ARCHIVE_NAME%.gz}" -C "target/${{ matrix.target }}/release" "${NAME}"
gzip -n -9 "${ARCHIVE_NAME%.gz}"

echo "BIN_PATH=${BIN_PATH}" >> $GITHUB_OUTPUT
echo "ARCHIVE_NAME=${ARCHIVE_NAME}" >> $GITHUB_OUTPUT
echo "BIN_PATH=${BIN_PATH}" >> "$GITHUB_OUTPUT"
echo "ARCHIVE_NAME=${ARCHIVE_NAME}" >> "$GITHUB_OUTPUT"

- name: Generate attestation for release binary
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
Expand All @@ -92,7 +104,7 @@ jobs:
- name: Upload release archive
env:
GH_TOKEN: ${{ github.token }}
run: gh release upload ${VERSION} ${{ steps.archive.outputs.ARCHIVE_NAME }}
run: gh release upload "${VERSION}" "${{ steps.archive.outputs.ARCHIVE_NAME }}"

build_xwin:
name: ${{ matrix.target }}
Expand Down
8 changes: 4 additions & 4 deletions .github/zizmor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ rules:
template-injection:
ignore:
- release_choco.yml
- release_github.yml:95
- release_github.yml:275
- release_github.yml:302
- release_github.yml:340
- release_github.yml:107
- release_github.yml:287
- release_github.yml:314
- release_github.yml:352
- release_sub_homebrew.yml:29
- release_sub_homebrew.yml:56
- release_sub_homebrew.yml:69
Expand Down
Loading