[CI] Update test fetcher script - #14607
Open
DN6 wants to merge 4 commits into
Open
Conversation
… with feature-split matrix output
…so the fetcher does not over-select through shared mixins
…an HF token, not an accelerator
…sformer_flux.py` instead of a separate `utils.py`
DN6
commented
Aug 26, 2026
| release_memory(pipe) | ||
|
|
||
| def test_not_empty_state_dict(self): | ||
| from diffusers import AutoPipelineForText2Image |
Collaborator
Author
There was a problem hiding this comment.
The fetcher decides "this test depends on file X" by looking at the imports at the top of the test file. AutoPipelineForText2Image lives in auto_pipeline.py, and that file imports every pipeline in the library. So if a test file imports AutoPipelineForText2Image at the top, the fetcher concludes that test depends on every pipeline, and any PR touching any pipeline would run it.
| assert False, "Parameters not the same!" | ||
|
|
||
| def test_local_files_only_with_sharded_checkpoint(self): | ||
| from diffusers.models import FluxTransformer2DModel |
Collaborator
Author
There was a problem hiding this comment.
Similar issue as AutoPipelines. Importing Flux and SD3 at the top of the file suggests to the test fetcher that these model objects are a dependency for anything that imports from this file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Migrating PR CI to use the test fetcher by default. Doing this in parts to keep review manageable and testable.
First step is to just update the fetcher script.
The test fetcher determines which tests to run based on which files have been changed in a PR. It works in the following way
diffusers/srcandtests/to build a reverse dependency map. This map is a dictionary from a file in our repo to every file that would be affected if the file were changed. It is built in the following way.pyundersrc/diffusers/andtests/return the files it imports. e.g.test_pipeline_flux.py -> [src/diffusers/pipelines/flux/pipeline_flux.py,...]test_pipeline_flux.py's list also containstransformer_flux.py, attention.pyand so on, because these are modules are imported bypipeline_flux.pysrc/diffusers/pipelines/flux/pipeline_flux.py -> [tests/pipelines/flux/test_pipeline_flux.py, tests/pipelines/flux/test_pipeline_flux_control.py, ...](LoraTesterMixin → lora, MemoryTesterMixin → memory, cpu_offload, group_offload, …). It uses this to construct a feature based matrix for pipeline/models tests so that the actual tests jobs run in feature based buckets. e.g model-core, model-memory, model-attention, pipeline-core, pipeline-lora. Here is an example of a workflow running this new test matrix: https://github.com/huggingface/diffusers/actions/runs/32945379597/job/98105816662Once the fetcher script is merged into
main, we can update the relevant workflows and test them side by side with the existing CI to ensure no gaps. FYI the fetcher tends to over select tests rather than miss them.