Skip to content

Maintenance: preserve selection policy and roll back failed updates - #40

Closed
tdejager wants to merge 36 commits into
RoboStack:mainfrom
tdejager:fix/maintenance-transactions
Closed

tdejager wants to merge 36 commits into
RoboStack:mainfrom
tdejager:fix/maintenance-transactions

Conversation

@tdejager

@tdejager tdejager commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

We should be able to retry a failed maintenance command without repairing the checkout first. A request for one distro also should not quietly change the package policy or pins of another.

Before

add-package could turn a conditional selection into an unconditional one. A pinning update could continue after rendering failed, or change shared pins while only updating one consumer. Repository checks rejected build number zero and empty migration lists. A failed new-distro command could leave a directory that blocked the next attempt.

After

Existing exclusions, skips and conditional selections are reported without broadening their scope. Pinning updates restore the previous configuration and rendered files on failure. A changed shared follower gets its own version and migration list in distro.yaml; the shared pins and other distros stay untouched. A no-op stays a shared follower.

Build number 0 and an explicit empty migration list are valid. New-distro creation validates its inputs before creating the destination and removes only the directory it created if generation fails, so the same command can be retried.

I opted for per-distro pin ownership here because the bot opens one PR per distro. Updating shared pins would need a coordinated rebuild plan for every consumer.

Testing

22 regression tests passed with this PR applied directly to main, including selection boundaries, pinning rollback and retryable distro creation. The empty-migration renderer fix and its test now live in this PR; that test uses Vinca's real renderer.

On the completed series, pixi run --locked rs add-package rosidlcpp jazzy --preview reported the existing exclusion and left the selection unchanged. pixi run --locked rs check passed for all four distros. No real snapshot or pin-version update was applied to the repository during these checks.

Closes #28
Closes #31
Closes #32
Closes #33

Independent PR

This PR is based directly on main and can merge independently of #39 (build/CI) and #41 (imports/naming). Its tests pass without either of those branches. Git merge checks verified all six merge orders: each combines without conflicts and produces the same intended tree. All 78 regression tests pass when the independent branches are combined.


PR description written by Astra on 2026-10-09. Sol reviewed these maintenance changes.

Follow-up updated by Astra on 2026-10-09. A separate gpt-6-astra:high agent reviewed all three PRs, found the additional auto-merge queue race in #39, and confirmed its fix. No actionable findings remained; the review was read-only.

Tobias-Fischer and others added 30 commits October 6, 2026 19:10
distros/<distro>/ holds only what is specific to a distribution; shared/ holds
the rosdep mapping, pinning and tests; tools/robostack.py runs every task for a
distribution (pixi run rs <distro> <task>); CI builds only the distributions a PR
touches. Data copied from RoboStack/ros-{rolling,humble,lyrical,kilted} main and
ros-jazzy codex/cross-distro-sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ings

shared/vinca.yaml (common settings and the 72 packages every distribution
selects), shared/pkg_additional_info.yaml (36 entries) and
shared/dependencies.yaml (61 entries) hold what all distributions agree on;
distros/<d>/ keeps the rest. tools/robostack.py assembles distros/<d>/work/
from both and runs every task there; skip_existing, conda_index and the
snapshot paths are derived. The full linux-64 recipe sets of rolling, humble
and lyrical are byte-identical before and after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anches

- tools/maintenance.py: check (sanity checks, run in CI), update-snapshot and
  find-stale per distribution, update-pinning (shared conda-forge pinning) and
  new-distro; parse-command for @robostack-bot comments and command issues.
- .github/workflows/bot.yml + issue template: weekly snapshot/pinning PRs,
  commands on demand.
- Staged builds: ROBOSTACK_UPLOAD_CHANNEL redirects uploads to a (test)
  prefix.dev channel, 'none' (default outside RoboStack) skips them;
  PUSH_BUILD_BRANCHES enables pushing build branches outside RoboStack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…round trip); check it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… work/); refuse the fallback

Without ./.scripts/build_win.bat in its working directory vinca-gha silently
falls back to an outdated built-in Windows script (boa build); found by pushing
a test build branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Share vinca.yaml, pkg_additional_info.yaml and dependencies.yaml settings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tools/maintenance.py add-package PKG... [DISTRO...] [--preview]: adds the
  packages (normalising ros2-/ros-<distro>- names) to the selection of every
  given distribution that has them, in sorted position, in shared/vinca.yaml if
  that's all of them; generates the linux-64 recipes and lists what will be
  built and what is already published as a dependency.
- Package request issue form (anyone): the bot replies with a preview;
  @robostack-bot add-package by a maintainer opens the PR, which closes the issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e, import tool

- main.yml pushes the staged build branches whenever the bot app (or GHA_PAT) is
  available, without an organisation check; the build workflows upload to each
  distribution's channel unless ROBOSTACK_UPLOAD_CHANNEL redirects them.
- robostack-bot add-package opens the PR for anyone (valid ROS package names
  only, at most 10 per request); the other commands stay with maintainers.
- tools/import_distro.py moves a distribution here from its own repository
  (re-importing lyrical reproduces distros/lyrical exactly).
- kilted keeps the pins its published packages were built with.
- README for users and maintainers, bug report template with a distribution field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…resh plotjuggler Windows patch for 3.17.2

- humble: same fix as rolling/lyrical (windows.h min/max macros break
  rclcpp's duration.hpp/time.hpp in influx_component).
- jazzy: the old Windows patch targeted an older plotjuggler (its protobuf
  changes are upstream now); rebased rolling's 3.17.2 Windows patch (Qt link
  libraries, plotjuggler_base exports) onto jazzy's release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…PI_KEY; don't build tl_expected/sophus under conda-forge names

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ing updates, fix linking the request

- update-rosdistro-snapshot and update-conda-forge-pinning bump build_number and the
  mutex minor version, drop per-package build numbers the new one catches up with, and
  (pinning) move plain mutex run_constraints to the new pins
- commands take several distributions or all; one job per distribution
- issue forms: no pre-filled title, multi-select distributions defaulting to all,
  add-package in the command form (anyone)
- fix the Link the request step (' #' started a YAML comment)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ci_config.py also matches plain (unprefixed) names, for packages vinca built under a
conda-forge name. Reset ci.yaml after merging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ebuilds and package selection; fold long bot reports

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…brary/autoware_trajectory; distro-specific ros2cli test; drop a cancelled run's last package before caching

- parameter_traits and generate_parameter_library 0.7.6: use conda-forge's tl-expected
  (tl_expected is mapped to cpp-expected); autoware_trajectory takes humble's patch
- ros2cli test: only rolling's and lyrical's patches install a fish completion
- testpr: a cancelled build can package an empty build (seen on Windows); remove the
  newest artifact before saving the cache; evict jazzy's empty rmw_fastrtps_shared_cpp

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…umber from the channel; jazzy/humble/kilted robot_state_publisher test

- pyqt-builder 1.20 (conda-forge, 2026-10-07) defaults to sip ABI 12.20/13.13, which
  needs sip >=6.16; qt_gui_cpp keeps sip <6.16 and failed with a missing sip.h.in
- update-rosdistro-snapshot is no longer scheduled; its bump sets build_number to the
  highest build of the distribution's released packages + 1 and removes all
  per-package build numbers
- robot_state_publisher osx test: CycloneDDS 0.10 (humble, jazzy, kilted) doesn't know
  allow_multicast; use ROS_LOCALHOST_ONLY there as before

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ected patches for autoware_downsample_filters and pick_ik

- update-rosdistro-snapshot snapshots the distribution's latest sync tag in
  ros/rosdistro (e.g. jazzy/2026-10-05) instead of rosdistro master, which includes
  releases that are only in ros-testing; the sync is recorded as rosdistro_sync in
  distro.yaml and the PR links the rosdistro changes and the Discourse announcement
- the bot checks daily for new syncs (rs rosdistro-syncs) and opens or updates one PR
  per distribution; rolling (rosdistro_sync: manual) keeps reading master on request
- vinca from Tobias-Fischer/vinca feat/snapshot-rosdistro-ref (vinca-snapshot
  --rosdistro-ref) until it is merged upstream

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All matrix jobs on a platform share the cache key, so they raced to save it
("Unable to reserve cache ... another job may be creating this cache").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/vinca#161)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…::tl_expected

conda-forge's libroboplan-core 0.7.0 build 3 links the system tl-expected
(open-planning/roboplan#369), so the bundled roboplan::tl_expected target is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ted first

Carry open-planning/roboplan-ros#85 for jazzy, lyrical and rolling: conda-forge's
libroboplan-core 0.7.0 build 3 doesn't export roboplan::tl_expected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in rolling

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…RoboStack#5)

Ready for RoboStack: production defaults, package requests from anyone, import tool
pixi's task shell re-parsed it, so an apostrophe in a package request broke the
plan step (issue RoboStack#9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
robostack-bot: pass the request text through the environment
…s2-* names, no kilted) (RoboStack#8)

See RoboStack#8 for the full description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ect (RoboStack#16)

See RoboStack#16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer and others added 5 commits October 9, 2026 15:06
…oboStack#17)

prefix.dev's Repository Access (OIDC) matches a GitHub workflow by exact file
name. Each build branch only carries its own workflow, so one fixed name lets
a single authorized source cover every distribution and platform. README:
how to set it up, and the canonical <namespace>/<channel> upload reference.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…oboStack#18)

The build workflows pass PREFIX_API_KEY from the (now deleted) secret, so the
variable is set but empty; rattler-build then refuses --generate-attestation
("cannot be used with an API key").

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oboStack#21)

With ROBOSTACK_UPLOAD_CHANNEL set (the test channel), the packages earlier
stages build are uploaded there, but rattler-build only searched the
distribution's release channel and conda-forge, so later stages failed to
resolve them (e.g. humble's yasmin_editor). Put that channel first; reruns
then also skip what is already uploaded.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…Stack#22)

prefix.dev's sharded repodata, which rattler reads, lists a new upload a few
minutes after repodata.json does. A stage starting right after the previous
one uploaded then fails with "No candidates were found" for a package that
is on the channel (lyrical linux-64, plansys2_pddl_parser). build-ci now
retries that failure twice, 3 minutes apart; other failures aren't retried.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@tdejager
tdejager force-pushed the fix/maintenance-transactions branch from 7caa4fb to 7dba8b5 Compare October 9, 2026 12:45
@Tobias-Fischer

Copy link
Copy Markdown
Collaborator

Thanks! Real bugs on main are fixed here and worth a small PR on their own:

  • build number 0 is rejected by check(), although new-distro writes it;
  • an empty migration list is rejected;
  • the return code of render-pinning is ignored.

I'd drop the rest:

  • _set_pinning always writes the pins into distro.yaml. After one daily bot cycle every follower owns its own pin, and shared/pinning/conda_forge.yaml is never used again. That works directly against the goal of one shared pinning for all distros.
  • add_package regresses. selected is now the union over all distros, so a humble if: win selection, or humble's if: not linux skip, blocks the package for every other distro. The PR's own test (test_maintenance.py:132) asserts this. Main resolves selections per distro via read_vinca.
  • Rollback and the new-distro rewrite: the bot's CI checkout is thrown away after a failed job, so this is a lot of code (and ~460 lines of mocked tests) for little gain.
  • CI and conflicts: CI didn't run. With Drop the distro mutex's run_constraints #23, 5 tests fail (their fixtures use mutex run_constraints, which Drop the distro mutex's run_constraints #23 forbids), and the README conflicts.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

@Tobias-Fischer

Copy link
Copy Markdown
Collaborator

This was closed automatically when we rewrote main's history into topic commits (GitHub closes PRs whose base history is replaced). The parts we're keeping from #39–#41 are in #46, with you as co-author. The reasons for leaving out the rest are in the review comments above. Thanks for the audit; it found several real bugs!

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants