Skip to content

vinca-gha: stage the generated recipes without resolving the distribution again - #172

Merged
Tobias-Fischer merged 2 commits into
RoboStack:masterfrom
Tobias-Fischer:perf/gha-skip-resolution
Oct 10, 2026
Merged

Tobias-Fischer merged 2 commits into
RoboStack:masterfrom
Tobias-Fischer:perf/gha-skip-resolution

Conversation

@Tobias-Fischer

@Tobias-Fischer Tobias-Fischer commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

vinca-gha called get_full_tree(), which regenerates the outputs of every package of the distribution (build_all, ~1900 packages for lyrical: package.xml downloads, SPDX license normalisation, mutex generation again), only to read their host/run dependency names. vinca (generate-recipes) has just written the recipes the workflow orders, so their own requirements.host/run are enough to build the graph. vinca.yaml is still read (cheaply) for ros_distro and the pixi versions.

Side effects:

  • Recipes the build_all tree lacked are no longer dropped. build_all uses the live rosdistro release packages, so a recipe for a snapshot package that is no longer in the index was silently left out of the workflow ("filter out packages that we are not actually building"). In lyrical that hit easynav_gps_localizer, roboplan_common, roboplan_core (and dependents of the latter were staged without waiting for them).
  • Dependencies through packages that are not rebuilt (already on the channel) no longer order the stages: if A -> B (published) -> C (rebuilt), A and C can now share a stage. A only needs the C build that B was built against, so this ordering wasn't needed; if B has to pick up the new C it is rebuilt too and then in the recipes.
  • Recipe files are read in sorted order, so the job order doesn't depend on the file system.

Measured locally (RoboStack monorepo, rs <distro> generate-recipes then rs <distro> gha --batch-size 25, vinca = current robostack-integration with/without this commit):

distro / platform recipes before after
lyrical linux-64 167 61.4 s 2.7 s
lyrical osx-arm64 109 60.2 s 2.3 s
jazzy linux-64 73 55.1 s 2.4 s
humble linux-64 31 179.5 s 1.8 s
lyrical linux-64, no skip_existing 2235 36.2 s 7.3 s

The workflow is not byte-identical: within a dependency stage the package order (and so which packages share a batch job) followed the topological order of the full tree and now follows the recipes. Every package's dependency stage is identical: same jobs and per-package stage for jazzy/humble linux-64 and lyrical osx-arm64; for lyrical linux-64 the only differences are the packages listed above that were previously dropped, and the roboplan packages that now wait for roboplan_core. The host/run dependency names read from the recipes equal those of the full tree for all 2229 common packages.

Tests: get_recipe_requirements and a main() test that stages recipes from ./recipes without resolving anything; full vinca suite passes (263 passed).

🤖 Generated with Claude Code

…tion again

vinca-gha regenerated the outputs of every package of the ROS distribution
(build_all, ~1900 packages: package.xml downloads, license normalisation)
only to read their host/run dependencies, although generate-recipes had just
written the recipes it orders. The stages only order those recipes, so their
own requirements are enough; vinca.yaml is still read for the distribution
and the pixi versions.

This also keeps recipes that the build_all tree lacked: it lists the live
rosdistro release packages, so a recipe from a snapshot package no longer in
the index (e.g. lyrical's easynav_gps_localizer, roboplan_core) was silently
left out of the workflow. The recipes are read in sorted order so the
workflow does not depend on the file system's directory order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@traversaro traversaro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GReat! After you fix the conflict, I guess this is ready to merge.

@Tobias-Fischer
Tobias-Fischer merged commit 1412ace into RoboStack:master Oct 10, 2026
3 checks passed
@Tobias-Fischer
Tobias-Fischer deleted the perf/gha-skip-resolution branch October 10, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants