release: generate release catalog companions for build artifacts

[msarahan]

Posted by Codex (GPT-5) on behalf of @msarahan. This pull request description is LLM-generated; readers should treat its content accordingly.

Why

RAPIDS currently creates a release by tagging repositories in dependency order and allowing those tags to trigger builds that publish directly to release channels. That makes the tag a prerequisite for producing the artifacts, forces downstream repositories to wait for upstream publication and CDN synchronization, and prevents us from validating the complete final-version artifact set before any part of it is public.

The proposed release platform reverses that relationship: build the complete release candidate first, verify the exact bytes as a set, and only then promote those approved bytes and tag their source commits. For that to work, the shared build workflows need a release-candidate mode that:

• produces final-version artifacts without pushing a tag;
• keeps candidate artifacts out of public package channels;
• records exactly what every matrix job built and the source revision that produced it; and
• places artifacts and their evidence in private storage where the release platform can collect them by frozen release train.

This PR supplies that integration for the standard Conda, wheel, and custom build workflows. It consumes the release catalog action developed in rapidsai/shared-actions#136.

Merge dependency

Important

rapidsai/gha-tools#274 must be merged before this PR. This PR now passes RAPIDS_BUILD_TYPE=release-candidate through unchanged and depends on gha-tools to recognize its release-version semantics while excluding the public rapidsai and rapidsai-nightly channels from candidate dependency resolution.

What changes

Release-candidate build mode

• Adds release-candidate as a supported build_type and uses the full nightly build matrix for candidate coverage.
• Accepts the frozen release-train SHA-256 and the final source tag selected by the release platform.
• Creates that final tag only in the local checkout and disables pushes to the source remote. This gives existing release-aware build tooling its expected version context without publishing the tag or coupling tag creation to a new build.
• Preserves established nightly dependency-resolution behavior while identifying the orchestration run as a release candidate.

Private staging instead of publication

• Candidate builds do not upload their package bundle to the normal GitHub artifact path.
• Conda and wheel publication workflows do not run for release-candidate builds, so candidates cannot enter Anaconda.org, PyPI, or other normal public destinations before approval.
• Each candidate build obtains short-lived AWS credentials through GitHub OIDC and uploads its artifacts and release catalog companion to the private rapids-release-candidates S3 bucket.
• Normal branch, pull-request, nightly, and release behavior remains on the existing paths.

Release catalog companions

For every release-candidate matrix job, the shared-actions release catalog action creates a companion folder containing:

• release-catalog-entries.json, with package identity, artifact SHA-256, source repository and commit, workflow/run identity, and evidence paths;
• one SLSA/in-toto provenance statement per artifact; and
• one CycloneDX 1.6 identity SBOM per artifact.

The generated CycloneDX document currently identifies the artifact itself and binds it to its digest. Component/dependency inventory is intentionally left for a future schema version.

Companions do not modify the artifacts they describe. S3 uploads are conditional and accept an existing object only when its bytes match, making retries idempotent while rejecting conflicting output for the same destination.

Standard and custom producers

• Conda C++, Conda Python, and wheel workflows configure companions automatically. They scan only their declared output directory, discover supported packages, and read identity from package metadata.
• Standard catalog keys are repository-level release-policy groups such as conda:<repository> and wheel:<repository>; they are not unique package IDs.
• custom-job.yaml remains opt-in through release-catalog-config. This supports outputs such as Maven JARs and standalone archives without treating every custom artifact as publishable release content.
• Conda packages, wheels, and Maven JARs can derive identity from embedded metadata. Other formats must provide an explicit package_identity_file; the action does not infer identity from arbitrary filenames.

Explicit, validated inputs

All producers pass the checked-out source revision explicitly as source-sha: ${{ env.RAPIDS_SHA }}. RAPIDS_SHA comes from the caller-selected checkout when supplied and otherwise from the checked-out repository, ensuring the catalog identifies what the job actually built.

The action validates configuration against its versioned input schema before inspecting artifacts and validates generated entries against its output schema. Configuration, materialization, and upload failures are surfaced as GitHub Actions error annotations.

See the shared-actions release catalog documentation for the companion layout, schemas, package parsing rules, evidence semantics, and S3 layout.

Release-platform flow

1. The release planner freezes repository commits, the dependency graph, versions, variants, and dependency locks into a release train.
2. The controller dispatches these shared workflows in the graph order recorded by that plan.
3. Each job builds final-version artifacts without a public tag and stages immutable artifacts plus its companion under the release-train identity in S3.
4. The release platform aggregates the job companions, verifies the expected inventory and checksums, and materializes the candidate for QA and scanning.
5. Approval and rollover promote the exact verified artifacts and create tags at the already-recorded source commits; rollover does not rebuild them.

Compatibility and rollout

• Existing non-candidate callers retain their current build and publication behavior.
• Standard Conda and wheel callers require no release-catalog configuration.
• Custom artifact producers participate only when they provide release-catalog-config.
• The shared-actions reference is pinned to the candidate implementation from PR Remove 11.2 and 11.5 tests #136 while the two PRs are developed and tested together; it should move to the merged immutable revision before this PR is finalized.

Tracks rapidsai/build-infra#381.

[msarahan]

Marking this as do not merge because we should change the shared-actions refs after merging rapidsai/shared-actions#136

I'll be out for a bit, so dismissing my blocking review to unblock you. I reviewed the shared-actions PR that complements this (rapidsai/shared-actions#136 (review)).

Can review this one if you need when I'm back.