Skip to content

Conversation

@neoblizz
Copy link
Member

No description provided.

@github-actions github-actions bot added in-progress We are working on it iris Iris project issue labels Nov 17, 2025
@neoblizz neoblizz marked this pull request as ready for review November 22, 2025 20:31
Copilot AI review requested due to automatic review settings November 22, 2025 20:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR revamps the testing infrastructure to improve test organization and execution. The key changes include consolidating workflow files, adding support for separate test directories (examples, unittests, ccl), and improving test isolation through explicit preamble calls in all_reduce tests.

Key Changes

  • Consolidated testing workflows into a single comprehensive iris-tests.yml file with a matrix strategy that tests each directory (examples, unittests, ccl) with different rank counts (1, 2, 4, 8)
  • Updated test scripts to accept a test directory parameter, enabling more granular test execution
  • Enhanced all_reduce tests to explicitly call preamble before running operations for better test isolation

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
.github/workflows/iris-tests.yml New consolidated workflow with matrix strategy for testing all directories across multiple GPU configurations
.github/workflows/iris-tests-apptainer.yml Deleted old Apptainer-specific workflow, functionality now in iris-tests.yml
.github/workflows/iris-pip-install-test.yml Deleted pip installation test workflow, consolidated into main testing workflow
.github/workflows/iris-performance-regression-test.yml Added DOCKER_IMAGE_NAME environment variable for consistency
.github/workflows/iris-external-validation-test.yml Added DOCKER_IMAGE_NAME environment variable for consistency
.github/scripts/run_tests.sh Enhanced to accept test directory parameter and validate directory existence
.github/scripts/container_run.sh Updated to use DOCKER_IMAGE_NAME variable with fallback to "iris-dev"
.github/scripts/container_exec.sh Updated to use DOCKER_IMAGE_NAME variable with fallback to "iris-dev"
.github/scripts/container_build.sh Updated to use DOCKER_IMAGE_NAME variable with fallback to "iris-dev"
tests/ccl/test_all_reduce.py Disabled "ring" variant testing and added explicit preamble calls for better test isolation

Comment on lines 76 to 77
if variant == "ring":
config.all_reduce_num_rings = min(2, config.comm_sms)
Copy link

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead code: The conditional block for the "ring" variant will never execute since "ring" has been commented out in the parametrize decorator above (line 19). Consider removing this block or uncommenting the "ring" variant in the test parameters.

Suggested change
if variant == "ring":
config.all_reduce_num_rings = min(2, config.comm_sms)

Copilot uses AI. Check for mistakes.
[
"atomic",
"ring",
# "ring",
Copy link

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing documentation: The "ring" variant is commented out without explanation. Consider adding a comment explaining why this variant is disabled (e.g., "# ring - disabled due to known issues" or "# ring - TODO: fix compatibility issues").

Suggested change
# "ring",
# "ring", # Disabled due to known issues with the ring algorithm (TODO: fix compatibility issues)

Copilot uses AI. Check for mistakes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like us to continue testing against the three different installs:

  • pip install git+https://github.com/${{ github.repository }}.git@${{ github.sha }}
  • pip install -e .
  • pip install .

Sometimes source structure and import assumptions are only captured in one vs the other.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mawad-amd why don't we just make sure the installs work there?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pip install without running the tests you mean?

Copy link
Collaborator

@mawad-amd mawad-amd Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are issues that won't show up unless we actually run the tests. If we really have to test against one and do some smoke tests against the other, then I would prefer we always do the full test suite against pip install git+https://github.com/${{ github.repository }}.git@${{ github.sha }}

Copy link
Collaborator

@mawad-amd mawad-amd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great and much cleaner. Thanks!

@neoblizz neoblizz merged commit cbec566 into main Dec 8, 2025
127 of 159 checks passed
@neoblizz neoblizz deleted the muhosama/testing-infra branch December 8, 2025 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

in-progress We are working on it iris Iris project issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants