Skip to content

Add action for creating release catalog companion - #136

Open
msarahan wants to merge 36 commits into
rapidsai:mainfrom
msarahan:agent/release-build-output-container
Open

Add action for creating release catalog companion#136
msarahan wants to merge 36 commits into
rapidsai:mainfrom
msarahan:agent/release-build-output-container

Conversation

@msarahan

@msarahan msarahan commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

This adds a schema-validated action that creates a release catalog companion for an existing build artifact. The release platform can aggregate these job-level entry files into the release catalog. That catalog supports the two purposes tracked in rapidsai/build-infra#381:

  • assembling releases that can be tested and verified before formal tagging
  • preserving build-time software evidence, especially for binary artifacts such as wheels where that evidence cannot reliably be reconstructed later.

The action:

  • validates configuration before inspecting artifacts
  • Generates a "companion" - a folder including release-catalog-entries.json (an artifact inventory) plus per-artifact SBOM and provenance files
  • uploads the "companion" to S3 where the release platform can find them

@msarahan msarahan added the DO NOT MERGE Development or canary PR; do not merge label Aug 7, 2026
@msarahan
msarahan force-pushed the agent/release-build-output-container branch from 76acdd5 to 3a9568f Compare August 7, 2026 18:55
@msarahan msarahan added improvement Improves an existing functionality non-breaking Introduces a non-breaking change and removed DO NOT MERGE Development or canary PR; do not merge labels Aug 13, 2026
@msarahan
msarahan marked this pull request as ready for review August 13, 2026 14:30
@msarahan
msarahan requested a review from a team as a code owner August 13, 2026 14:30
@msarahan
msarahan requested a review from jameslamb August 13, 2026 14:30
@msarahan
msarahan force-pushed the agent/release-build-output-container branch 4 times, most recently from 4144565 to a18a4a7 Compare August 13, 2026 20:09
@msarahan msarahan added breaking Introduces a breaking change non-breaking Introduces a non-breaking change and removed non-breaking Introduces a non-breaking change breaking Introduces a breaking change labels Aug 13, 2026
@msarahan msarahan changed the title Make release output dispatch container-safe Simplify and validate release build-output action configuration Aug 17, 2026

@jameslamb jameslamb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did my best to review this.

My read is that this replaces a bunch of individual arguments with:

  • automatically calculating some of the values
  • allowing a few more values only needed in specific cases (like cuVS Java builds) to be passed in as a single dictionary via this new config: argument

At a surface level that sounds good to me! Less opportunity for misconfiguration across the repos, easier to change the behavior in the future.

My comments in this PR are light, but see my review on rapidsai/shared-workflows#609 ... I think documentation (ideally written by a human) in the release-build-output/ directory would be really helpful for folks to understand why this exists and what it does. I looked through the code, logs, and output from the testing PRs (rapidsai/dask-cuda#1672, rapidsai/nx-cugraph#274, NVIDIA/cuvs#2400) and think I have a picture of that, but I personally would find it hard to modify this setup if a new requirement came in.

Comment thread .pre-commit-config.yaml Outdated
Comment thread release-build-output-dispatch/action.yml Outdated
Comment thread release-build-output/config.schema.json Outdated
@msarahan msarahan changed the title Simplify and validate release build-output action configuration Add schema-validated release catalog action Aug 18, 2026
@msarahan msarahan changed the title Add schema-validated release catalog action Add schema-validated release catalog companion action Aug 18, 2026

@jameslamb jameslamb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes, but I'm really struggling to thoughtfully review this.

I really did try but it's time for me to stop.

Put some suggestions below for things that would have helped me to provide a thoughtful review, I hope you'll consider them for the next round of development on this.


This PR is large and complicated, most of the code is uncommented.

As an example, release-catalog/materialize.sh seems like a critical part of the release-catalog action to me, but it's 400+ lines of bash and jq with only 2 code comments, one of which is a TODO filled with jargon I don't understand. More comments, especially top-level ones with description and usage in the larger scripts, would have helped.


The release-catalog README reads as a literal description of the code as-is, not something I could use to form expectations about how the software's supposed to work at a high level.

There's heavy use of jargon throughout and I don't know what is your terminology and what is coming from standards (like SPDX) that we have to meet. Either way, I don't understand many of the terms.

example: what the difference between a "build identity", "identity evidence", and "job-level envelope"? Are any of those synonymous with "companion"?.


I'm struggling to understand what this is expected to do, so I'm struggling to review how well it's doing it.

If we were to take the final "assembled" output for 1 repo, say NVIDIA/cuvs, how large would that be? If it's just a bunch of JSON files and not too large, could they be checked into source control here or somewhere else so we could see an example of what the end state is that we're trying to get to? That'd be really helpful.


The links in the description to tracking issues that just contain huge walls of seemingly unedited LLM output did not at all help my understanding.

Descriptions of this work written by you would probably have helped my understanding.


I left a few other suggestions on the diff. I feel strongly about separating all this new testing code into its own job instead of jamming it onto the existing checks: job that by convention we use for static analysis like pre-commit, not unit tests.

Otherwise, I'm clicking "approve" to say "ok, fine, go ahead and experiment with this". You want to do this this way and this is an area you have ownership over, I'm getting out of the way.

Comment thread .github/workflows/pr.yml
Comment thread release-catalog/README.md Outdated
Comment thread release-catalog/README.md Outdated
Comment thread release-catalog/README.md Outdated
Comment thread release-catalog/materialize.sh
Comment thread release-catalog/config.schema.json Outdated
],
"properties": {
"release_catalog_key": {
"description": "Release-catalog entry name that owns these artifacts. Default for conda & wheels is <ecosystem>:<repository-name>, such as conda:cudf. Custom jobs specify their own, such as maven:cuvs-java.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a tension in this line that I wondered throughout reviewing this... what does this field actually do and what uniqueness guarantees does it need to meet?

We build multiple conda packages from the cudf repo, so <ecosystem>:<repository-name> seems to not offer enough uniqueness.

Or if it does and it's important that the repository something was built from be part of this key (for provenance reasons) then why would cuvs-java not be maven:cuvs? cuvs-java is built from the cuvs repo.

@msarahan msarahan changed the title Add schema-validated release catalog companion action Add action for creating release catalog companion Sep 1, 2026
@msarahan
msarahan force-pushed the agent/release-build-output-container branch from c432a91 to 02bead3 Compare September 2, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants