Skip to content

Drop the distro mutex's run_constraints - #23

Closed
Tobias-Fischer wants to merge 36 commits into
mainfrom
drop-mutex-constraints
Closed

Tobias-Fischer wants to merge 36 commits into
mainfrom
drop-mutex-constraints

Conversation

@Tobias-Fischer

Copy link
Copy Markdown
Collaborator

As discussed with @traversaro: the mutex only keeps one ROS distribution per environment. Library versions come from the packages' own dependencies, and wrong metadata is fixed in the package itself (a patch, pkg_additional_info.yaml, or a conda-forge repodata patch), not in the mutex.

Why: mutex run_constraints apply to every user environment, also to libraries the user's packages never use. When another package needs a different version, solvers silently fall back to old ROS builds. For example, ros-base + libprotobuf 6.*:

with constraints without
jazzy mutex 0.15, rclcpp 28.1.18 mutex 0.19, rclcpp 28.1.22
lyrical mutex 0.18, rclcpp 32.0.0 mutex 0.21, rclcpp 32.0.3

Verification: I solved desktop + perception + ros_gz for every distribution on linux-64, osx-arm64 and win-64, with the published mutex and with an unconstrained copy of it:

  • humble, lyrical: identical environments. The shared libboost/pcl/libprotobuf/vtk pins and humble's krb5 were redundant, since the packages already pin these themselves.
  • jazzy, rolling: newer builds (openvino 2026.4, ffmpeg 9.0.2, newer OpenCV builds). The constraints added to work around solver trouble only held these back now. I installed the jazzy environment on osx-arm64 and ran: OpenCV 5 with dnn, cv_bridge, rclpy, C++ talker → listener, Gazebo Harmonic, and gz-msgs/gz-transport with libprotobuf 7.35.1.

Changes:

  • shared/vinca.yaml: run_constraints: [] (vinca requires the key) and a comment stating the policy. The per-distribution constraints are removed.
  • Each distribution publishes a new build of its mutex at the same version (mutex_package.build_number = build_number + 1), so no other package needs a rebuild. This needs Mutex: honour mutex_package.build_number; run_constraints optional vinca#169: vinca never generated such a build before. vinca is pinned to f9d68bb on robostack-integration. With that pin, lyrical's mutex recipe is number: 28, string: lyrical_28, run_constraints: [].
  • rs check rejects mutex run_constraints, so they can't come back as a quick fix.
  • Pinning bot: the mutex constraint sync and the mutex rebuild are gone.
  • Docs: README and the robostack-versions skill.
  • First unit tests of the tooling: pixi run test (pytest, tools/tests), run in the PR check job.
    • They cover the guard, every distribution's assembled config, and the partial rebuild leaving the mutex alone.
    • Re-adding a constraint makes them fail.

Follow-up: check_dependency_compat.py still has its mutex-constraint code (PIN MISMATCH, --mutex-only). With an empty list it is a no-op; I'll remove it separately.

🤖 Generated with Claude Code

Tobias-Fischer and others added 30 commits October 6, 2026 19:10
distros/<distro>/ holds only what is specific to a distribution; shared/ holds
the rosdep mapping, pinning and tests; tools/robostack.py runs every task for a
distribution (pixi run rs <distro> <task>); CI builds only the distributions a PR
touches. Data copied from RoboStack/ros-{rolling,humble,lyrical,kilted} main and
ros-jazzy codex/cross-distro-sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ings

shared/vinca.yaml (common settings and the 72 packages every distribution
selects), shared/pkg_additional_info.yaml (36 entries) and
shared/dependencies.yaml (61 entries) hold what all distributions agree on;
distros/<d>/ keeps the rest. tools/robostack.py assembles distros/<d>/work/
from both and runs every task there; skip_existing, conda_index and the
snapshot paths are derived. The full linux-64 recipe sets of rolling, humble
and lyrical are byte-identical before and after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anches

- tools/maintenance.py: check (sanity checks, run in CI), update-snapshot and
  find-stale per distribution, update-pinning (shared conda-forge pinning) and
  new-distro; parse-command for @robostack-bot comments and command issues.
- .github/workflows/bot.yml + issue template: weekly snapshot/pinning PRs,
  commands on demand.
- Staged builds: ROBOSTACK_UPLOAD_CHANNEL redirects uploads to a (test)
  prefix.dev channel, 'none' (default outside RoboStack) skips them;
  PUSH_BUILD_BRANCHES enables pushing build branches outside RoboStack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…round trip); check it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… work/); refuse the fallback

Without ./.scripts/build_win.bat in its working directory vinca-gha silently
falls back to an outdated built-in Windows script (boa build); found by pushing
a test build branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Share vinca.yaml, pkg_additional_info.yaml and dependencies.yaml settings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tools/maintenance.py add-package PKG... [DISTRO...] [--preview]: adds the
  packages (normalising ros2-/ros-<distro>- names) to the selection of every
  given distribution that has them, in sorted position, in shared/vinca.yaml if
  that's all of them; generates the linux-64 recipes and lists what will be
  built and what is already published as a dependency.
- Package request issue form (anyone): the bot replies with a preview;
  @robostack-bot add-package by a maintainer opens the PR, which closes the issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e, import tool

- main.yml pushes the staged build branches whenever the bot app (or GHA_PAT) is
  available, without an organisation check; the build workflows upload to each
  distribution's channel unless ROBOSTACK_UPLOAD_CHANNEL redirects them.
- robostack-bot add-package opens the PR for anyone (valid ROS package names
  only, at most 10 per request); the other commands stay with maintainers.
- tools/import_distro.py moves a distribution here from its own repository
  (re-importing lyrical reproduces distros/lyrical exactly).
- kilted keeps the pins its published packages were built with.
- README for users and maintainers, bug report template with a distribution field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…resh plotjuggler Windows patch for 3.17.2

- humble: same fix as rolling/lyrical (windows.h min/max macros break
  rclcpp's duration.hpp/time.hpp in influx_component).
- jazzy: the old Windows patch targeted an older plotjuggler (its protobuf
  changes are upstream now); rebased rolling's 3.17.2 Windows patch (Qt link
  libraries, plotjuggler_base exports) onto jazzy's release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…PI_KEY; don't build tl_expected/sophus under conda-forge names

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ing updates, fix linking the request

- update-rosdistro-snapshot and update-conda-forge-pinning bump build_number and the
  mutex minor version, drop per-package build numbers the new one catches up with, and
  (pinning) move plain mutex run_constraints to the new pins
- commands take several distributions or all; one job per distribution
- issue forms: no pre-filled title, multi-select distributions defaulting to all,
  add-package in the command form (anyone)
- fix the Link the request step (' #' started a YAML comment)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ci_config.py also matches plain (unprefixed) names, for packages vinca built under a
conda-forge name. Reset ci.yaml after merging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ebuilds and package selection; fold long bot reports

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…brary/autoware_trajectory; distro-specific ros2cli test; drop a cancelled run's last package before caching

- parameter_traits and generate_parameter_library 0.7.6: use conda-forge's tl-expected
  (tl_expected is mapped to cpp-expected); autoware_trajectory takes humble's patch
- ros2cli test: only rolling's and lyrical's patches install a fish completion
- testpr: a cancelled build can package an empty build (seen on Windows); remove the
  newest artifact before saving the cache; evict jazzy's empty rmw_fastrtps_shared_cpp

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…umber from the channel; jazzy/humble/kilted robot_state_publisher test

- pyqt-builder 1.20 (conda-forge, 2026-10-07) defaults to sip ABI 12.20/13.13, which
  needs sip >=6.16; qt_gui_cpp keeps sip <6.16 and failed with a missing sip.h.in
- update-rosdistro-snapshot is no longer scheduled; its bump sets build_number to the
  highest build of the distribution's released packages + 1 and removes all
  per-package build numbers
- robot_state_publisher osx test: CycloneDDS 0.10 (humble, jazzy, kilted) doesn't know
  allow_multicast; use ROS_LOCALHOST_ONLY there as before

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ected patches for autoware_downsample_filters and pick_ik

- update-rosdistro-snapshot snapshots the distribution's latest sync tag in
  ros/rosdistro (e.g. jazzy/2026-10-05) instead of rosdistro master, which includes
  releases that are only in ros-testing; the sync is recorded as rosdistro_sync in
  distro.yaml and the PR links the rosdistro changes and the Discourse announcement
- the bot checks daily for new syncs (rs rosdistro-syncs) and opens or updates one PR
  per distribution; rolling (rosdistro_sync: manual) keeps reading master on request
- vinca from Tobias-Fischer/vinca feat/snapshot-rosdistro-ref (vinca-snapshot
  --rosdistro-ref) until it is merged upstream

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All matrix jobs on a platform share the cache key, so they raced to save it
("Unable to reserve cache ... another job may be creating this cache").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/vinca#161)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…::tl_expected

conda-forge's libroboplan-core 0.7.0 build 3 links the system tl-expected
(open-planning/roboplan#369), so the bundled roboplan::tl_expected target is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ted first

Carry open-planning/roboplan-ros#85 for jazzy, lyrical and rolling: conda-forge's
libroboplan-core 0.7.0 build 3 doesn't export roboplan::tl_expected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in rolling

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#5)

Ready for RoboStack: production defaults, package requests from anyone, import tool
pixi's task shell re-parsed it, so an apostrophe in a package request broke the
plan step (issue #9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
robostack-bot: pass the request text through the environment
…s2-* names, no kilted) (#8)

See #8 for the full description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ect (#16)

See #16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer and others added 6 commits October 9, 2026 15:06
)

prefix.dev's Repository Access (OIDC) matches a GitHub workflow by exact file
name. Each build branch only carries its own workflow, so one fixed name lets
a single authorized source cover every distribution and platform. README:
how to set it up, and the canonical <namespace>/<channel> upload reference.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
)

The build workflows pass PREFIX_API_KEY from the (now deleted) secret, so the
variable is set but empty; rattler-build then refuses --generate-attestation
("cannot be used with an API key").

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

With ROBOSTACK_UPLOAD_CHANNEL set (the test channel), the packages earlier
stages build are uploaded there, but rattler-build only searched the
distribution's release channel and conda-forge, so later stages failed to
resolve them (e.g. humble's yasmin_editor). Put that channel first; reruns
then also skip what is already uploaded.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The mutex only keeps one ROS distribution per environment. Its run_constraints
applied to every user environment, also to libraries the user's packages
don't use, and made solvers silently fall back to old ROS builds when another
package needed a different version (ros-base + libprotobuf 6: jazzy resolved
to mutex 0.15 / rclcpp 28.1.18 instead of 0.19 / 28.1.22). Library versions
come from the packages' own dependencies; wrong metadata gets fixed in the
package (patch, pkg_additional_info.yaml, conda-forge repodata patch).

Checked by solving desktop + perception + ros_gz for every distribution on
linux-64, osx-arm64 and win-64 with and without them: humble and lyrical
resolve to identical environments, jazzy and rolling to newer
openvino/opencv/ffmpeg builds (jazzy's smoke-tested on osx-arm64: OpenCV,
cv_bridge, rclpy, talker/listener, gz).

- shared/vinca.yaml: run_constraints: [] with the policy; distros: removed,
  and a new mutex build at the same version (mutex_package.build_number)
- rs check / tools/tests: reject mutex run_constraints
- pinning bot: no more mutex constraint sync or mutex rebuild
- pixi run test: first unit tests of the tooling, run in PR CI

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without it the new mutex builds are never generated (RoboStack/vinca#169).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant