ci: cancel merge queue runs on the first job failure

[andygrove]

What is the problem the feature request solves?

A merge queue run keeps every other job running after one job has already failed, even though the failure alone is enough to remove the pull request from the queue.

ci.yml has no fail-fast across jobs. The only required check is the aggregate required_checks job, which runs with if: always() and reports only after every upstream job has completed. So the queue does not learn about a failure until the whole run finishes, and every job that was still running or queued at the moment of the first failure is wasted: the group is dequeued regardless of how those jobs end, and the re-queued pull request runs all of them again.

Concrete example from 2026-09-15, the queue run for #5939 (run 34993857297):

• 16:26 UTC: the PyArrow UDF (Spark 4.2) job failed on a transient 429 downloading the Maven distribution.
• 17:26 UTC, an hour later: the run was still in progress with 9 jobs running (Spark 4.0 and 4.1 SQL shards and two macOS builds), 91 jobs already passed.
• The run had no way to succeed after 16:26, so roughly an hour of multi-runner wall clock was spent producing results the queue would discard. The group queued behind it (fix: align time parsing and native second extraction with Spark #5738) also had to be rebuilt and rerun.

GitHub Actions does not retry a failed job on its own, and a failed job cannot be re-run while the run is still in progress. Once the run completes and the pull request is dequeued, re-running the failed job no longer helps; the pull request has to be re-queued and every job runs again from scratch.

Describe the potential solution

Cancel the whole merge_group workflow run as soon as any job in it fails, so the pull request is dequeued immediately and the runners go back to the pool.

Two ways to express this:

1. A cancel step in each reusable workflow / composite, run with if: failure() && github.event_name == 'merge_group', that calls the cancel API on ${{ github.run_id }} (for example gh run cancel with actions: write). This fires from inside the failing job, so it reacts within seconds.
2. A watcher pattern is not enough on its own: a job that reads the needs context only runs after its dependencies complete, so it cannot short-circuit them.

Restrict it to merge_group. On pull_request runs, seeing every failure in one run is useful to the author; in the queue, the only question is pass or fail, and the first failure answers it.

Note that cancelled already counts as a failure in required_checks, so the aggregate job needs no change for the queue to dequeue the group.

Trade-offs to weigh:

• A second, independent failure in the same run becomes invisible until the next attempt. In the queue that cost is small because a re-queued run will surface it anyway.
• Transient failures such as the Maven 429 above should be retried inside the job (see ci: retry the Maven distribution download in every job that runs mvnw #5881) rather than by cancelling, so the two changes are complementary: retry the flake, cancel on a real failure.

Additional context

• required_checks in .github/workflows/ci.yml and its comment block describe the current pass/fail folding.
• ci: next set of jobs to move from the PR tier to the merge queue tier #5870 has the broader CI tier analysis and runner-hour numbers.
• ci: retry the Maven distribution download in every job that runs mvnw #5881 adds retry to the Maven bootstrap that caused the failure in the example above.