Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions .github/actions/build-native-ci/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: Build or restore the Linux CI native library
description: 'Reuse an exact-input native library, otherwise build it with the CI profile'
runs:
using: composite
steps:
- name: Pin native build flags
shell: bash
run: echo 'RUSTFLAGS=-Ctarget-cpu=x86-64-v3 -Clink-arg=-fuse-ld=bfd' >> "$GITHUB_ENV"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This swaps a step-level env: for a job-scoped one. The value is right: all four callers declare a workflow-level RUSTFLAGS: "-Clink-arg=-fuse-ld=bfd", and GITHUB_ENV does win over it, because the runner evaluates workflow and job env once into the same dictionary that GITHUB_ENV later writes to (JobExtension.InitializeJob populating Global.EnvironmentVariables, then StepsRunner seeding each step's env context from it).

What changes is scope. -Ctarget-cpu=x86-64-v3 now applies to every later step in the caller's job, where the old step-level env: applied to the one cargo build. Nothing downstream invokes Cargo today, so this is not a bug, but it is an unflagged widening. Dropping this step and putting env: RUSTFLAGS: ... on the fingerprint and build steps restores the old scope and removes a step.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scope observation is correct. I kept the single definition because Andy explicitly requested removing the duplicated fingerprint/build literals in the previous review. These are native-producer jobs, and their remaining steps upload or package the existing library without invoking Cargo. There is no affected downstream build today. Reintroducing the two copies would reverse that agreed simplification, so I have left this unchanged and am keeping the thread open for discussion.


# Call after checkout and setup-builder. Compute once, before Cargo writes
# generated Rust files, and use the same keys for both restore and save.
- name: Fingerprint native build inputs
id: key
shell: bash
run: python3 dev/ci/native-cache-key.py --profile ci --github-output "$GITHUB_OUTPUT"

- name: Restore native library cache
id: binary-cache
uses: actions/cache/restore@v6
with:
path: native/target/ci/libcomet.so
key: ${{ steps.key.outputs.binary-key }}
# Main restores the incremental cache below, including libcomet.so.
# A lookup avoids downloading the same library separately.
lookup-only: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}

- name: Restore incremental Cargo cache
id: cargo-cache
if: steps.binary-cache.outputs.cache-hit != 'true' || (github.event_name == 'push' && github.ref == 'refs/heads/main')
uses: actions/cache/restore@v6
with:
path: native/target
key: ${{ steps.key.outputs.source-key }}
restore-keys: ${{ steps.key.outputs.restore-prefix }}

- name: Build native library (CI profile)
if: steps.binary-cache.outputs.cache-hit != 'true' || (github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.cargo-cache.outputs.cache-hit != 'true')
shell: bash
run: |
cd native
# A library miss always builds, even on an exact incremental hit.
# Main also builds to populate a missing incremental entry; two exact
# hits already supply the library and leave neither cache to publish.
cargo build --locked --profile ci

- name: Save native library cache
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.binary-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v6
with:
path: native/target/ci/libcomet.so
key: ${{ steps.key.outputs.binary-key }}

- name: Save incremental Cargo cache
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.cargo-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v6
with:
path: native/target
key: ${{ steps.key.outputs.source-key }}
36 changes: 36 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,42 @@ entry through `restore-keys` and downloads whatever else it needs, which is
what a cold pull request already did. See the push-tier discussion above for
which jobs do run on main and therefore do write.

## Reusing Linux native builds

The Linux, Spark SQL, Iceberg and manual writer workflows call
`.github/actions/build-native-ci` after checkout and `setup-builder`. PR, queue,
scheduled and manual runs restore `native/target/ci/libcomet.so` and skip Cargo
on an exact library match. Only pushes to `main` save caches. Main skips Cargo
when both the library and incremental cache match exactly, and builds when either
lacks an exact match to replenish it.
An incremental cache hit alone never replaces compilation. Builds use
`cargo build --locked --profile ci`; manifest changes requiring a lockfile update
must include that update to `native/Cargo.lock`. Artifact paths remain unchanged.

`dev/ci/native-cache-key.py` snapshots native sources, protobufs, dependencies,
Cargo configuration and shared build/setup actions before source generation.
It includes Rust versions, installed package versions, architecture, JDK
release/path, and Cargo/Rust, C/C++ compiler/flag and HDFS environment overrides.
Caller workflows are excluded because their selected tools and environment are
observed directly. Spark edits, documentation, generated files and disabled
contrib sources preserve the key; contrib manifests remain inputs for `--locked`.
Benchmarks enter only the debug key. The input lists and glob matcher are shared
with main's routing in `compute-changes.py`; code generation uses `x86-64-v3`.

The helper supports the official Rust container and `setup-builder`. Introducing
external tools or files requires updating that contract: an override's path does
not identify arbitrary contents stored there. Both binary and incremental keys
retain package and JDK identity because native dependencies compile against JNI
headers and link `libjvm`, and Cargo does not fully track external tool/header
changes. Unrelated package updates can therefore cause conservative misses.

The CI and debug incremental caches hold only `native/target`, including compiled
dependencies. Cargo fetches registry and Git dependency sources as needed; those
downloads are not duplicated in the repository's limited cache storage. Fallback
restores permit source changes within the same dependency/build environment.
Rust checks and tests always run with their separate debug cache. Preflight checks
fingerprint invalidation, main's routing, and the action's cache-hit/miss behavior.

## Retrying flaky network operations

**Maven.** `.mvn/maven.config` tunes the Maven Resolver HTTP transport: six
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ jobs:
- name: Check Iceberg shard inventory validation
run: python3 dev/ci/test-iceberg-shards.py

- name: Check native caching
run: |
python3 dev/ci/test-native-cache-key.py
python3 dev/ci/test-native-cache-workflow.py

- name: Check CI config invariants
run: python3 dev/ci/check-ci-config.py

Expand Down
29 changes: 2 additions & 27 deletions .github/workflows/iceberg_spark_test_reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,33 +81,8 @@ jobs:
id: shards
run: python3 dev/ci/check-iceberg-shards.py --github-output "$GITHUB_OUTPUT"

- name: Restore Cargo cache
uses: actions/cache/restore@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
restore-keys: |
${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-

- name: Build native library
# Use CI profile for faster builds (no LTO) and to share cache with pr_build_linux.yml.
run: |
cd native && cargo build --profile ci
env:
RUSTFLAGS: "-Ctarget-cpu=x86-64-v3 -Clink-arg=-fuse-ld=bfd"

- name: Save Cargo cache
uses: actions/cache/save@v6
if: github.ref == 'refs/heads/main'
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
- name: Build or restore native library
uses: ./.github/actions/build-native-ci

- name: Upload native library
uses: ./.github/actions/upload-artifact-retry
Expand Down
73 changes: 16 additions & 57 deletions .github/workflows/pr_build_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -369,27 +369,10 @@ jobs:
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ env.RUST_VERSION }}
jdk-version: 17 # JDK only needed for JVM module proto generation
jdk-version: 17 # JNI headers and libjvm are native build inputs.

- name: Restore Cargo cache
uses: actions/cache/restore@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
restore-keys: |
${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-

- name: Build native library (CI profile)
run: |
cd native
# CI profile: same overflow behavior as release, but faster compilation
# (no LTO, parallel codegen)
cargo build --profile ci
env:
RUSTFLAGS: "-Ctarget-cpu=x86-64-v3 -Clink-arg=-fuse-ld=bfd"
- name: Build or restore native library
uses: ./.github/actions/build-native-ci

- name: Upload native library
uses: ./.github/actions/upload-artifact-retry
Expand All @@ -398,18 +381,6 @@ jobs:
path: native/target/ci/libcomet.so
retention-days: 1

- name: Save Cargo cache
uses: actions/cache/save@v6
# The push run is the cache warmer (see the header); a scheduled run
# at the same sha would only re-archive an entry that already exists.
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}

# Run Rust tests (runs in parallel with build-native, uses debug builds).
# Owns main's `cargo-debug` cache entry, so it runs in cache-refresh-only mode
# as well. The tests themselves are a small slice of its runtime -- almost all
Expand All @@ -432,17 +403,19 @@ jobs:
rust-version: ${{ env.RUST_VERSION }}
jdk-version: 17

- name: Fingerprint Rust test build inputs
# Default HDFS support compiles against JNI headers and links libjvm.
# Cached build-script outputs also contain JDK paths (core/build.rs).
id: cargo-key
run: python3 dev/ci/native-cache-key.py --profile debug --github-output "$GITHUB_OUTPUT"

- name: Restore Cargo cache
id: cargo-cache
uses: actions/cache/restore@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
# Note: Java version intentionally excluded - Rust target is JDK-independent
key: ${{ runner.os }}-cargo-debug-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
restore-keys: |
${{ runner.os }}-cargo-debug-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-
path: native/target
key: ${{ steps.cargo-key.outputs.source-key }}
restore-keys: ${{ steps.cargo-key.outputs.restore-prefix }}

- name: Rust test steps
uses: ./.github/actions/rust-test
Expand All @@ -451,13 +424,10 @@ jobs:
uses: actions/cache/save@v6
# The push run is the cache warmer (see the header); a scheduled run
# at the same sha would only re-archive an entry that already exists.
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.cargo-cache.outputs.cache-hit != 'true'
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-debug-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
path: native/target
key: ${{ steps.cargo-key.outputs.source-key }}

linux-test:
# `lint` is already upstream via build-native; it is listed here so this
Expand Down Expand Up @@ -643,17 +613,6 @@ jobs:
# Download to release/ since Maven's -Prelease expects libcomet.so there
path: native/target/release/

# Restore cargo registry cache (for any cargo commands that might run)
- name: Restore Cargo registry
uses: actions/cache/restore@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
key: ${{ runner.os }}-cargo-registry-${{ hashFiles('native/**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-registry-

- name: Java test steps
uses: ./.github/actions/java-test
with:
Expand Down
29 changes: 2 additions & 27 deletions .github/workflows/spark_sql_test_reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,33 +97,8 @@ jobs:
rust-version: ${{ env.RUST_VERSION }}
jdk-version: ${{ inputs.java }}

- name: Restore Cargo cache
uses: actions/cache/restore@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
restore-keys: |
${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-

- name: Build native library (CI profile)
run: |
cd native
cargo build --profile ci
env:
RUSTFLAGS: "-Ctarget-cpu=x86-64-v3 -Clink-arg=-fuse-ld=bfd"

- name: Save Cargo cache
uses: actions/cache/save@v6
if: github.ref == 'refs/heads/main'
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
- name: Build or restore native library
uses: ./.github/actions/build-native-ci

- name: Upload native library
uses: ./.github/actions/upload-artifact-retry
Expand Down
19 changes: 2 additions & 17 deletions .github/workflows/spark_sql_writer_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,23 +83,8 @@ jobs:
rust-version: ${{ env.RUST_VERSION }}
jdk-version: ${{ steps.resolve.outputs.java }}

- name: Restore Cargo cache
uses: actions/cache/restore@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-${{ hashFiles('native/**/*.rs') }}
restore-keys: |
${{ runner.os }}-cargo-ci-${{ hashFiles('native/**/Cargo.lock', 'native/**/Cargo.toml') }}-

- name: Build native library (CI profile)
run: |
cd native
cargo build --profile ci
env:
RUSTFLAGS: "-Ctarget-cpu=x86-64-v3 -Clink-arg=-fuse-ld=bfd"
- name: Build or restore native library
uses: ./.github/actions/build-native-ci

- name: Stage native library at release path
run: |
Expand Down
6 changes: 5 additions & 1 deletion dev/ci/check-ci-config.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@
[".github/actions/maven-bootstrap/action.yaml"],
{"build_linux", "build_linux_full", "build_linux_all_profiles"},
),
# Linux's compact native cache is shared by every Spark/Iceberg producer.
# Helper edits also match macOS's broad dev/ci filter, as before.
([".github/actions/build-native-ci/action.yaml"], BUILD_JOBS - {"build_macos"}),
(["dev/ci/native-cache-key.py"], BUILD_JOBS),
# Spot checks that the additions above did not widen unrelated routes.
(["docs/source/user-guide/overview.md"], {"docs"}),
(["native/core/benches/parquet_read.rs"], {"benchmark"}),
Expand Down Expand Up @@ -399,7 +403,7 @@
CACHE_REFRESH_WORKFLOW = WORKFLOWS / "pr_build_linux.yml"
CACHE_REFRESH_JOBS = {
"lint": "gates build-native and linux-test-rust, and costs 40 seconds",
"build-native": "writes the cargo-ci cache (native/target, CI profile)",
"build-native": "writes the compact native library and cargo-ci caches",
"linux-test-rust": "writes the cargo-debug cache (native/target, debug)",
"verify-benchmark-results-tpch": "writes the TPC-H SF=1 dataset and java-maven caches",
"verify-benchmark-results-tpcds": "writes the TPC-DS SF=1 dataset and java-maven caches",
Expand Down
Loading
Loading