fix: fail fast in RayProcess with TaskGroup

[toby-coleman]

Summary

A failing component could leave RayProcess waiting for every remote call, including blocked downstream components. Use asyncio.TaskGroup to propagate failures early and explicitly request cancellation of remote calls.

Changes

• Add a typed coroutine adapter for Ray ObjectRefs and local awaitables, preserving result order and grouped exceptions. Cancel submitted Ray calls on group failure or caller cancellation.
• Replace gather_except throughout RayProcess and replace the blocking ray.get used to update cancellation status with an async wait.
• Limit attribute refreshes during error propagation to five seconds and log refresh errors without masking the original failure.
• Add regression tests for local/remote failure, remote cancellation, caller cancellation, mixed results, and failed or unresponsive attribute refreshes. Update the existing failure test for early sibling cancellation.
• Document cooperative remote cancellation and potentially stale component attributes while remote cleanup is still running.

[github-actions]

Benchmark comparison for 3817b9c7 (base) vs 52973154 (PR)

------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.14-64bit/0001_pr)         406.6837 (1.0)      415.5747 (1.00)     411.2808 (1.0)      3.3230 (2.01)     412.1818 (1.0)      4.2374 (1.93)          2;0  2.4314 (1.0)           5           1
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.14-64bit/0001_base)     410.9802 (1.01)     415.2527 (1.0)      412.7132 (1.00)     1.6512 (1.0)      412.7010 (1.00)     2.1998 (1.0)           2;0  2.4230 (1.00)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5297315423

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Mark cancelled Ray components as stopped

When one component call fails, _gather only cancels the sibling ObjectRefs; a component sets its status to RUNNING before step() or run(), and Ray cancellation exits without invoking Component.cancel() or another terminal status transition. Consequently, the cancelled actors and RayStateBackend can remain permanently RUNNING after the process becomes FAILED—the modified integration test now skips the previous STOPPED assertion for exactly this path. Trigger best-effort status cleanup for the cancelled component actors while preserving fail-fast propagation.

AGENTS.md reference: AGENTS.md:L26-L28

Useful? React with 👍 / 👎.

[codecov]

Codecov Report

❌ Patch coverage is 94.28571% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
plugboard/process/ray_process.py	94.28%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[toby-coleman]

Remove this from the docs

[Copilot]

🟢 Approval recommended

The implementation is consistent with the stated behavior and includes targeted regression coverage.

Pull request overview

Updates RayProcess to fail fast and cooperatively cancel outstanding Ray calls.

Changes:

• Introduces TaskGroup-based ordered gathering and cancellation.
• Bounds attribute refreshes during failure handling.
• Adds unit/integration coverage and cancellation documentation.

File summaries

File	Description
plugboard/process/ray_process.py	Implements fail-fast task coordination.
tests/unit/test_ray_process.py	Tests ordering, failures, and cancellation.
tests/integration/test_ray_process_taskgroup.py	Verifies cancellation with Ray actors.
tests/integration/test_process_with_components_run.py	Updates failure expectations.
docs/api/process/process.md	Documents cancellation behavior.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.