Repository navigation
refactor(comfyui): split runtime selection, process lifecycle and install into child modules - #884
Conversation
…i/runtime_env.rs comfyui.rs mixes install, process lifecycle, runtime selection and environment, and status. This moves the runtime selection (select_runtime and its default/selector/ready-runtime helpers and their error messages) and the child-process environment builders (runtime_environment_*, apply_runtime_environment and the path helpers) into a child module, with the tests that exercise them. The types they return stay in comfyui.rs. Pure move: no behaviour change; items are pub(super) only where comfyui.rs still calls them. The shared test fixtures (test_paths, ready_runtime_manifest, write_runtime_manifest) stay in comfyui.rs's tests module and are widened to pub(super) so the moved tests can reach them. docs/architecture.md describes the new module. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Moves start, stop, the background spawn (a detached child on Linux, a hidden PowerShell launcher on Windows), the running-state evaluation that status uses, the process probe and termination helpers, and the reinstall guard (running_from, refuse_while_running, running_advice) into a child module, with the test that exercises evaluate_running_state. comfyui.rs re-exports start and stop, so callers still write comfyui::start and comfyui::stop. Pure move: no behaviour change. None of the moved items is cfg-gated (the Windows paths are selected at run time), so the Linux build compiles them all. The saved state and its types stay in comfyui.rs. process imports runtime_env and the parent's types and helpers, and nothing else. docs/architecture.md and the path in rocm-core's quoting module doc describe the new location. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Moves the install action (including its dry run and the reinstall guard call sites), the source download and unpack, the requirements filtering, the torch-stack pinning and the logged uv command runner into a child module, together with the install unit tests and the Unix-only archive_properties property tests for the source unpack. comfyui.rs re-exports install, so callers still write comfyui::install. Pure move: no behaviour change; the manifest, the log paths and the GPU probe stay in comfyui.rs. install imports runtime_env and, for the running-ComfyUI guard, process; neither imports it. The fixtures the install tests share with the tests left in comfyui.rs stay there and are widened to pub(super). The two spinner-label entries in xtask's ASCII output contract and the cargo test filter in docs/testing.md follow the code to the new path; docs/architecture.md describes the module. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The architecture note for comfyui/process.rs now says it also uses comfyui/source_swap.rs, and cites that file as a link. Two comments in the e2e suite that described code now in comfyui/install.rs name that file. Comment and doc text only. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the ComfyUI module reorganization.
No open issues 🎉
What changed in this PR
Splits ComfyUI runtime selection, process lifecycle, and installation into focused child modules, continuing the CLI’s modularization without intended user-visible changes.
Changes:
- Moves implementation and associated tests into three child modules while preserving existing caller entry points.
- Updates architecture documentation and test filters for the new module layout.
- Adjusts spinner-label source paths and comment-only references to relocated code.
| File | Description |
|---|---|
xtask/src/ascii_output_contract.rs |
Points spinner-label exemptions at the installation module. |
tests/e2e-cucumber/tests/e2e/comfyui_steps.rs |
Updates an installation-path reference in a comment only. |
tests/e2e-cucumber/src/bin/fake-python.rs |
Updates comments describing Python probe locations. |
docs/testing.md |
Adds the relocated installation tests’ filter. |
docs/architecture.md |
Documents the child modules and their responsibilities. |
crates/rocm-core/src/quoting.rs |
Updates the Windows launcher path in a doc comment. |
apps/rocm/src/comfyui/runtime_env.rs |
Houses runtime selection, environment setup, and associated tests. |
apps/rocm/src/comfyui/process.rs |
Houses process lifecycle, running-state checks, and associated tests. |
apps/rocm/src/comfyui/install.rs |
Houses installation, dependency handling, and associated unit and property tests. |
Review effort: Balanced
juhovainio
left a comment
There was a problem hiding this comment.
Reviewed the full diff. The ComfyUI split into runtime_env, process and install is a pure move: comparing removed and added lines shows only visibility (pub(super)), imports, re-exports, module docs, test-fixture visibility and path text changed, and no function body was altered. The new module edges (install -> process/runtime_env, process -> runtime_env, runtime_env -> parent) match the docs, have no cycle, and the comfyui::{start,stop,install} re-exports keep callers unchanged. Skipping a new scenario is reasonable for a behavior-preserving refactor.
Non-blocking:
- The
#[cfg(unix)]gating on test-only imports could not be cross-compiled for Windows, so the Windows CI lane is the real check for unused-import lints; worth confirming that lane is green before merge. - One doc-comment nit inline.
| //! `-c <script>` for the pre-install torch-stack version probe (answered here | ||
| //! `apps/rocm/src/comfyui/install.rs` and `apps/rocm/src/comfyui.rs` shell out | ||
| //! to the runtime's Python twice: `-c <script>` for the pre-install torch-stack | ||
| //! version probe (in `install.rs`; answered here |
There was a problem hiding this comment.
Nit: the reworded sentence now has a mid-sentence parenthesis ("(in install.rs; answered here ...") that reads awkwardly. Consider separating the probe attribution from the answered-with clause.
There was a problem hiding this comment.
Thanks, reworded in 3522979: the comment now describes the two Python calls as separate sentences, naming install.rs for the pre-install torch-stack probe and comfyui.rs for the post-install GPU check, with what this binary answers to each. It is a comment-only change. The same push merges main into the branch, because #557 had reworded the #[ignore] messages of the property tests that this PR moves into install.rs; the new wording is applied there.
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review (Claude Code) · 1a71afa
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (commits 718881e, 3dd878b, afe37c2, 1a71afa; base 97d2e155). No ticket named; the PR implements a pure, behavior-preserving split of apps/rocm/src/comfyui.rs into comfyui/runtime_env.rs, comfyui/process.rs and comfyui/install.rs.
Blocking
-
Two self-hosted E2E checks are failing at the reviewed commit —
E2E tests (Strix Halo, Ubuntu, release, current)andE2E tests (Strix Halo, Ubuntu, nightly, current)(workflow run 41845673323). Both fail the same scenario,serve-40 - Serving a model with no automation enabled starts no background helper. The captured log shows the failure is in the unrelatedlemonademodel-serving path: anHTTP error 404fetching a third-partyllama.cpprelease build (lemonade-sdk/llama.cpp) plus a GPU "NOT drained" carryover from a prior test — nothing infeatures/model_serving.feature/serving_steps.rstouches any file this PR changes (onlycomfyui.rsand its three new child modules). This reads as environment/external-dependency flakiness rather than a regression from this diff, but per this repo's own CI-ownership policy ("a red check means 'not ready' until resolved"; handle flakes by rerun or maintainer escalation with evidence) it remains a blocking item until rerun or explicitly triaged, not something to wave through silently.
Confidence 100 · architectural · Fix: rerun the self-hosted E2E lane; if it reproduces as the known external-download/GPU-drain flake, escalate/track it separately from this PR rather than blocking on it indefinitely. -
The PR has a live merge conflict against
main(mergeable: CONFLICTINGas of this review). Root cause, confirmed independently viagit log/git cherryagainstorigin/main: PR #557 ("property-test--gpuselection…") already merged and reworded the#[ignore]messages on the samearchive_propertiesproperty tests this PR relocates intocomfyui/install.rs, so both touch the same lines in different files/locations. The author's own PR description already names this exact conflict in advance, so it isn't a surprise to anyone reading it — it just needs to actually be resolved before merge.
Confidence 100 · mechanical · Fix: rebase onto currentmainand reapply the#[ignore]-message wording from #557 insideinstall.rs.
Non-blocking
- Two self-hosted GPU E2E lanes (
MI300X, current,W7900, current) never completed for the reviewed commit (stuckQUEUED, apparently on runner availability rather than a failure). Nothing failed, so this isn't scored as blocking, but that coverage is simply unverified as of this review — worth confirming green before merge rather than assuming it's covered elsewhere.
Confidence 85 · architectural · Fix: re-check these two checks before merging; re-run if they remain stuck.
Decisions for the author
None.
Positive signals
- Every one of the three extraction commits is independently a byte-for-byte code move: diffing each commit's removed lines in
comfyui.rsagainst the added lines in its destination module (ignoring only visibility-keyword and import-line changes) gives ≥94.5% raw line-identity in every commit, and the only semantic differences are addedpub(super)on items a sibling module now calls, new module-doc comments, and reorganizeduseblocks. No logic, message, default, or condition drifted during the move. - The resulting module dependency graph is genuinely acyclic and matches what
docs/architecture.mdclaims:runtime_envimports no sibling,processimports onlyruntime_env(+source_swap),installimports bothruntime_envandprocess, and nothing imports back up into a child that would create a cycle. - Every incidental reference this split forces —
docs/architecture.md's module list,docs/testing.md's cargo test filters, the two pinned spinner-label paths inxtask/src/ascii_output_contract.rs, a doc comment incrates/rocm-core/src/quoting.rs, and two explanatory comments in the e2e suite — was updated accurately in the same change rather than deferred, and all of it checks out against the actual code. - The PR description's own "Notes for reviewers" section correctly anticipated the exact merge conflict with #557 before it happened, including which lines would collide.
Deployment notes
None — pure internal module split, no behavior, API, CLI output, or dependency change.
What this covered
Read completely and independently verified (line-by-line diffing of every moved function, import-graph tracing, and cross-checking every doc/comment edit against the real file contents it describes): apps/rocm/src/comfyui.rs (both pre- and post-split versions), the three new modules comfyui/runtime_env.rs, comfyui/process.rs, comfyui/install.rs, the unchanged sibling comfyui/source_swap.rs, crates/rocm-core/src/quoting.rs, docs/architecture.md, docs/testing.md, tests/e2e-cucumber/src/bin/fake-python.rs, tests/e2e-cucumber/tests/e2e/comfyui_steps.rs, xtask/src/ascii_output_contract.rs, and this repo's AGENTS.md. This was done as four independent fan-out passes (one per new module plus one cross-cutting pass for wiring/docs/AGENTS.md) run in parallel and blind to the PR description, consolidated with the orchestrating pass's own direct verification (byte-level move diffing, CI log retrieval, and git log/git cherry history analysis for the merge-conflict root cause). All four fan-out passes and the orchestrator's own checks returned clean (no ≥70-confidence code defects).
Did not run: a standalone "prior review comments on these files" pass (out of scope for a blind review) and an independent per-commit build of each of the four commits in isolation — the PR's own CI only builds/tests the squashed head, so the author's claim that each commit builds and passes tests on its own was not independently re-verified. Not reconciled against this PR's existing review discussion or comments.
🤖 Generated with Claude Code
…tall
The one conflict was in comfyui.rs: main reworded three #[ignore] messages in the archive_properties tests ("known defect" to "known gap"), and those tests now live in comfyui/install.rs. The new wording is applied there, and comfyui.rs keeps no archive_properties module.
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Comment text only: the probe attribution and the answered-with clause are now separate sentences. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Summary
apps/rocm/src/comfyui.rshad grown to about 5,000 lines, 2,300 of them production code, because it held four separate concerns in one file: choosing which ROCm runtime ComfyUI runs against, the ComfyUI process lifecycle, the install (source fetch, dependency install, GPU check), and status/log rendering with the manifest and state files. A reader looking for one of them had to scroll past the others, and every change to any of them touched the same file. This is part of the ongoing split ofapps/rocmmodules by seam. It moves the first three concerns into child modules ofcomfyuiand leaves the types, constants, paths, manifest and state I/O,status/logsrendering and the probe/capture helpers incomfyui.rs, which drops to about 1,460 lines including its tests.This is a pure move. No function body, message, flag or output changes, and every test keeps its name. There is one commit per module, in dependency order, and each commit builds and passes its tests on its own. A fourth, docs-only commit corrects the module notes and two comments in the e2e suite that named the old file. A merge of
mainfollows: the one conflict was the three#[ignore]messages in thearchive_propertiestests that #557 reworded fromknown defecttoknown gap, which now live incomfyui/install.rs, so the new wording is applied there andcomfyui.rskeeps no such module. A last small commit reflows the comment at the top oftests/e2e-cucumber/src/bin/fake-python.rsso each probe and its answer are separate sentences (comment text only).What moved
comfyui.rscomfyui/runtime_env.rsselect_runtime,format_available_runtime_keys,format_runtime_statuses,select_default_runtime,select_single_ready_runtime,select_runtime_by_selector,runtime_environment_for_manifest,runtime_environment_from_runtime,collect_rocm_runtime_paths,infer_python_env_root,apply_runtime_environment,runtime_environment_assignments,prepend_env_paths,split_runtime_paths,join_runtime_paths,push_existing_path,same_path_textcomfyui/process.rsstart,ensure_source_is_startable,running_from,refuse_while_running,running_advice,stop,spawn_comfyui_backgroundand its_unixand_windowsvariants,evaluate_running_state,wait_for_running_state,comfyui_run_state_cli_label,endpoint_is_reachable,process_is_running,windows_process_is_running,unix_process_is_running,wait_until_stopped,terminate_process_tree,terminate_windows_process_tree,terminate_unix_processcomfyui/install.rsinstall,swap_changes_summary,download_and_extract_source,unpack_source_archive,first_child_dir,download_file,filtered_requirement_specs,requirement_tokens,requirement_package_name,uv_install_args,TORCH_STACK_PACKAGES,TORCH_STACK_VERSION_PROBE,torch_stack_constraints,probe_torch_stack_versions,torch_constraint_lines,write_torch_constraints,run_uv_logged_command,OutputTarget,stream_logged_outputcomfyui.rsre-exportsstart,stopandinstall, somain.rsand the other callers still writecomfyui::start,comfyui::stopandcomfyui::installand need no change. The types (ComfyUiManifest,ComfyUiState,SelectedRuntime,ComfyUiRuntimeEnvironment, the run-state types),comfyui_source_archive_url(bothe2e-test-hooksvariants), the paths, manifest and state I/O,status_summary,render_status,render_chat_status_answer, the logs code,interrupted_reinstall_advice, and the probe/capture helpers (probe_comfyui,capture_configured_command, the temp-dir helpers,quote_log_arg,child_path_string) stay where they were. Items arepub(super)only where the parent or a sibling calls them.Tests moved with the code they exercise. The runtime-selection and environment tests went to
runtime_env, theevaluate_running_statetest toprocess, and the install tests plus the Unix-onlyarchive_propertiesproperty tests (ignored by default, as before) toinstall. Four tests spanstart,statusandinstalland stay incomfyui.rs:start_refuses_an_interrupted_reinstall_until_install_finishes_itand the threechat_status_answer_*tests. The fixtures that both the parent's tests and the install tests use (test_paths,ready_runtime_manifest,write_runtime_manifest,reinstall_fixture, the planted-release helpers and a few more) stay in the parent'stestsmodule aspub(super), and the child test modules import them from there.Because the tests moved, their full paths changed (for example
comfyui::tests::reinstall_*is nowcomfyui::install::tests::reinstall_*), so the cargo test filter indocs/testing.mdis updated to includecomfyui::install. The two spinner-label entries inxtask/src/ascii_output_contract.rs(Resolving and installing packages with uv…andFetching ComfyUI source archive…) now point atapps/rocm/src/comfyui/install.rs, because those literals moved. A doc comment incrates/rocm-core/src/quoting.rsthat names the Windows launcher's file now namescomfyui/process.rs.docs/architecture.mdhas an entry for each new module and an updatedcomfyui.rsentry.Module edges
No cycles:
runtime_envimports nothing from its siblings,processimports onlyruntime_env(and usessource_swap, which imports nothing fromcomfyui), andinstallimportsruntime_envandprocess; none of them is imported back. Each child also uses the parent's types and helpers, which is the normal parent/child shape thatsource_swapalready has.runtime_env:crate::runtimes_cmd::runtime_usability_status,crate::therock. Its tests also importcrate::comfyui::render_statusand the shared fixtures fromcrate::comfyui::tests.process: nocrate::edges beyond the parent;super::runtime_env::{apply_runtime_environment, runtime_environment_assignments, runtime_environment_for_manifest, same_path_text}, andsource_swap(a child ofcomfyuithat imports nothing fromcomfyui) for the interrupted-reinstall check inensure_source_is_startable.install:crate::cli_progress::AnimatedSpinner,crate::freeform_planner::format_structured_tool_call,crate::therock;super::process::{refuse_while_running, running_advice, running_from}andsuper::runtime_env::{apply_runtime_environment, runtime_environment_from_runtime, select_runtime}. Its tests also importcrate::comfyui::process::ensure_source_is_startable(Unix only) and the shared fixtures; the property-test module usescrate::archive_props.comfyui.rsno longer importscrate::cli_progress,crate::freeform_plannerorcrate::runtimes_cmd.Verification
Run locally on Linux, against the final tip, with a private target directory:
cargo fmt --all --checkandcargo clippy -p rocm -p xtask -p rocm-core --all-targets -- -D warnings: clean (also clean after each commit on its own).cargo test -p rocm -p rocm-core -p xtask --no-fail-fast: 2,797 passed, 0 failed, 10 ignored.cargo test -p rocm --bin rocm -- --listand-- --list --ignored, taken onorigin/mainbefore the first edit and again after each commit: empty each time (1,450 tests, 7 ignored, including the fourarchive_propertiescases that are ignored by default).cargo test -p rocm --bin rocm comfyui(88 passed, 4 ignored, same as before the move),cargo test -p rocm --bin rocm advised_commands,cargo test -p xtask ascii,cargo xtask manifest --checkandcargo xtask check-crate-edges: pass.status_reports_stopped_when_saved_comfyui_pid_is_gonefailed once in a parallel run and passed on every rerun (three alone, once in the fullcomfyuifilter). The test picks a free port by binding and releasing one, so another test's listener can take the same port; the test body is unchanged by this PR.Not run:
cargo xtask e2eandscripts/smoke_local.py(CI carries them), the Windows target. A cross-check with--target x86_64-pc-windows-msvccannot run here because the C toolchain thataws-lc-sysneeds for that target is not installed. The Windows lane is the check. To reduce what it has to catch: none of the moved production code iscfg(windows)-gated (the Windows spawn, process-probe andtaskkillpaths are selected at run time withruntime_is_windows()), so the Linux build compiles every one of them, and I re-read the Windows-specific functions inprocess.rsfor missing imports. The part the Linux build cannot see is test-only imports that are used only bycfg(unix)tests; I gated those imports with#[cfg(unix)]so they are not unused on Windows (ensure_source_is_startable,load_manifest,plant_cached_source_archive,plant_installable_release).User-visible behaviour
None, so no new scenario. This is a refactor with no change to any command, output, exit code, file or engine selection. The existing
comfyui-*scenarios intests/e2e-cucumber/features/comfyui.featureexercise the moved code through the built binary and are unmodified; CI runs them.Notes for reviewers
Read the first three commits in order; each is a mechanical move and can be checked with
git diff --color-moved. The only edits besides the moves are the imports each file needs,pub(super)on items that cross a module boundary, the re-export lines incomfyui.rs, the new module doc comments, and the docs and path pins listed above.The fourth commit changes the path text in two comments in
tests/e2e-cucumber/src/bin/fake-python.rsandtests/e2e-cucumber/tests/e2e/comfyui_steps.rsthat described code now incomfyui/install.rs. It is comment-only: no scenario, step or tag changes.#557 has merged; the proptest regression files under
apps/rocm/proptest-regressionsare keyed to other modules and none namescomfyui, and thearchive_propertiestests do not persist failures at all (failure_persistenceis off in the shared runner), so the module path change does not affect any regression lookup. Other open PRs that touch these files: #563 editsinstalland the interrupted-reinstall advice incomfyui.rs, so it will need itsinstallhunks reapplied incomfyui/install.rsafter commit 3 (and likely conflicts textually with commits 2 and 3, nearinterrupted_reinstall_adviceand at the end of the test module). Thedocs/architecture.md,docs/testing.mdandxtask/src/ascii_output_contract.rsedits can conflict with other PRs that touch the same lists.