-
Notifications
You must be signed in to change notification settings - Fork 225
Explore self-contained Java classifier JARs with a statically linked libcuopt #1818
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ramakrishnap-nv
wants to merge
35
commits into
main
Choose a base branch
from
java-static-classifiers
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
c156526
Make cuopt_static buildable for packaging, and add a scoped static build
ramakrishnap-nv bdb8905
Load the native library from the JAR, and package classifier JARs
ramakrishnap-nv e521ce7
Produce a working self-contained classifier JAR
ramakrishnap-nv 2f626af
Build and verify the classifier JAR in CI
ramakrishnap-nv a0816cb
Run java-static-build on pull requests
ramakrishnap-nv 87b17bb
Upload the classifier JARs as one Maven-repository-layout artifact
ramakrishnap-nv 6df31e9
Build every classifier and gather them, following cuDF's layout
ramakrishnap-nv a4219ed
Run the whole Java suite against each classifier JAR
ramakrishnap-nv a623cc4
Add java-static-build-matrix to the pr-builder aggregator
ramakrishnap-nv 78822d7
Route the new Maven calls through the retry wrapper
ramakrishnap-nv 0c0fceb
Route Java console logging through System.out instead of native stdout
ramakrishnap-nv 74325ec
Fix pre-commit findings: copyright year and clang-format alignment
ramakrishnap-nv 9722612
Merge branch 'main' into java-static-classifiers
ramakrishnap-nv 0bfbf4d
Patch vendored PSLP to respect verbose=false for its infeasible message
ramakrishnap-nv 9558f83
Strip trailing whitespace from the PSLP patch file (pre-commit)
ramakrishnap-nv 1ee37cb
Drop the trailing blank context line from the PSLP patch (pre-commit)
ramakrishnap-nv 072308e
Pin PSLP past v0.0.11 to the merged fix commit instead of patching
ramakrishnap-nv 59cc144
Merge the duplicate maven-surefire-plugin declaration into one
ramakrishnap-nv 6f37f3d
Add console-log-sink diagnostics; temporarily skip non-Java CI for fa…
ramakrishnap-nv 4bc2d27
Package cuDSS's OpenMP threading-layer plugin; dump Surefire fork-cra…
ramakrishnap-nv d468b48
Remove debug instrumentation and pr.yaml CI trimming used to root-cau…
ramakrishnap-nv fc6443a
Drop stale "Exploratory" framing; fix a misplaced job comment in buil…
ramakrishnap-nv 88b49bd
Add java/cuopt/.mvn/maven.config to reduce Maven Central rate-limiting
ramakrishnap-nv 2dad31c
Move java-static-test off conda onto rapidsai/ci-wheel; temporarily t…
ramakrishnap-nv f373279
Only attach source/javadoc jars when actually packaging, not on every…
ramakrishnap-nv 458b062
Merge remote-tracking branch 'origin/main' into java-static-classifie…
ramakrishnap-nv c1f9a15
Fix arm64 libcublas UnsatisfiedLinkError; correct the Maven retry pro…
ramakrishnap-nv 0002a06
Thread matrix.CUDA_VER into java-static-build's container env
ramakrishnap-nv ee920a8
Use a CUDA-version-pinned ci-conda image tag instead of an env-var ov…
ramakrishnap-nv ed2fc5e
Prefer the GCS Maven Central mirror over repo.maven.apache.org
ramakrishnap-nv c57b227
Merge remote-tracking branch 'origin/main' into java-static-classifie…
ramakrishnap-nv 2101490
Clean up leftover debug/duplicate content from the merge
ramakrishnap-nv 651e791
Address CodeRabbit review findings
ramakrishnap-nv 073d518
Merge remote-tracking branch 'origin/main' into java-static-classifie…
ramakrishnap-nv b7497f1
Simplify NativeLibraryLoader's cache-reuse check back to a size compa…
ramakrishnap-nv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| #!/bin/bash | ||
|
|
||
| # SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # Builds a self-contained Java classifier JAR and checks that it is actually self-contained. | ||
| # | ||
| # Unlike ci/build_java.sh, which installs a prebuilt libcuopt and links it as a shared library, | ||
| # this compiles libcuopt from source as a static archive and embeds it, so the JAR is the only | ||
| # thing a consumer installs. See #1817. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| if [[ -e /opt/conda/etc/profile.d/conda.sh ]]; then | ||
| . /opt/conda/etc/profile.d/conda.sh | ||
| fi | ||
|
|
||
| rapids-logger "Configuring conda strict channel priority" | ||
| conda config --set channel_priority strict | ||
|
|
||
| rapids-logger "Generating Java static build dependencies" | ||
| ENV_YAML_DIR=$(mktemp -d) | ||
| rapids-dependency-file-generator \ | ||
| --output conda \ | ||
| --file-key java_static \ | ||
| --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee "${ENV_YAML_DIR}/env.yaml" | ||
|
|
||
| rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n java_static | ||
|
|
||
| # Temporarily allow unbound variables for conda activation. | ||
| set +u | ||
| conda activate java_static | ||
| set -u | ||
|
|
||
| rapids-print-env | ||
|
|
||
| export CUOPT_PREFIX="${CONDA_PREFIX}" | ||
| STATIC_BUILD_DIR="${PWD}/cpp/build-static" | ||
| JNI_BUILD_DIR="${PWD}/java/cuopt/build/native-static" | ||
| JAR_OUTPUT_DIR="${PWD}/java/cuopt/classifier-jars" | ||
|
|
||
| rapids-logger "Building the scoped static libcuopt" | ||
| BUILD_DIR="${STATIC_BUILD_DIR}" bash java/cuopt/ci/build_static_libcuopt.sh | ||
|
|
||
| rapids-logger "Linking libcuopt into cuopt_jni" | ||
| cmake -S java/cuopt -B "${JNI_BUILD_DIR}" -GNinja \ | ||
| -DCMAKE_BUILD_TYPE=Release \ | ||
| -DCUOPT_PREFIX="${CUOPT_PREFIX}" \ | ||
| -DCUOPT_STATIC_BUILD_DIR="${STATIC_BUILD_DIR}" \ | ||
| -DCUOPT_EXTRA_INCLUDE_DIRS="${PWD}/cpp/include;${STATIC_BUILD_DIR}/include" | ||
| cmake --build "${JNI_BUILD_DIR}" --parallel "${PARALLEL_LEVEL:-$(nproc)}" | ||
|
|
||
| rapids-logger "Packaging the classifier JAR" | ||
| bash java/cuopt/ci/build_cuopt_java_jar.sh \ | ||
| --native-lib "${JNI_BUILD_DIR}/libcuopt_jni.so" \ | ||
| --cuda-version "${RAPIDS_CUDA_VERSION}" \ | ||
| --output-dir "${JAR_OUTPUT_DIR}" | ||
|
|
||
| # The JAR looking fine on this machine proves nothing: the build environment supplies every | ||
| # dependency by construction. This resolves them the way a consumer's machine would. | ||
| rapids-logger "Verifying the JAR is self-contained" | ||
| CLASSIFIER_JAR=$(find "${JAR_OUTPUT_DIR}" -name 'cuopt-*.jar' \ | ||
| ! -name '*-sources.jar' ! -name '*-javadoc.jar' -print -quit) | ||
| bash java/cuopt/ci/verify_jar_dependencies.sh --jar "${CLASSIFIER_JAR}" | ||
|
|
||
| # The gather job combines the classifier directories from every matrix entry into one Maven | ||
| # repository layout; this job uploads its own directory as-is. | ||
| rapids-logger "Result" | ||
| du -h "${CLASSIFIER_JAR}" | sed 's/^/ /' | ||
| find "${JAR_OUTPUT_DIR}" -type f | sed "s|^${JAR_OUTPUT_DIR}/| |" | sort |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: NVIDIA/cuopt
Length of output: 35959
Security Misconfiguration (CWE-494): Download of Code Without Integrity Check
Reachability: Internal · Exploitability: Difficult
Pin reusable workflows to immutable commits.
Replace the
@mainreferences at lines 70, 85, and 118 with reviewed commit SHAs. The static build and test jobs inherit secrets and requestid-token: write.📍 Affects 1 file
.github/workflows/build.yaml#L70-L70(this comment).github/workflows/build.yaml#L85-L85.github/workflows/build.yaml#L118-L118🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not addressed: all 17 other references to rapidsai/shared-workflows reusable workflows in this file already use @main, matching the convention every other RAPIDS repo (cuDF, cuVS, kvikio) uses for these same workflows. Pinning only these 3 new lines to a SHA would be inconsistent with the rest of the file without meaningfully improving security, since the other 17 would remain floating.