fix: only require barrier mode if more than one partition is trained on - #1912
Closed
Markus Cozowicz (eisber) wants to merge 2 commits into
Closed
fix: only require barrier mode if more than one partition is trained on#1912Markus Cozowicz (eisber) wants to merge 2 commits into
Markus Cozowicz (eisber) wants to merge 2 commits into
Conversation
Collaborator
Author
|
/azp run |
|
Hey Markus Cozowicz (@eisber) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report
@@ Coverage Diff @@
## master #1912 +/- ##
==========================================
- Coverage 86.81% 86.27% -0.55%
==========================================
Files 301 301
Lines 15783 15783
Branches 848 842 -6
==========================================
- Hits 13702 13616 -86
- Misses 2081 2167 +86
... and 10 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Scott Votaw (svotaw)
approved these changes
Apr 12, 2023
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Rana Singh (ranadeepsingh)
added a commit
to ranadeepsingh/SynapseML
that referenced
this pull request
Aug 1, 2026
## Summary Use barrier execution only when VowpalWabbit training enables it and the prepared dataset has more than one partition. Add Spark-stage execution-path assertions for enabled multi-partition, disabled multi-partition, and enabled single-partition training. ## Prompting Intent Revalidate the current VowpalWabbitBaseLearner behavior, restore the minimal source-compatible rule proposed by the ancient PR, prove it through TDD and targeted VW validation, and keep the change isolated from LightGBM. ## Linked Sources - Original proposal: microsoft#1912 - Initial CI failure: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=229206650 - Repository review guidance: .github/skills/code-review/SKILL.md - Local validation guidance: .github/skills/synapseml-local-setup/SKILL.md ## Rationale Keep the public API and multi-partition synchronization behavior unchanged with a short-circuit partition-count guard. Observe Spark stage metadata rather than adding a test-only PipelineStage subclass, and use a marker job to drain asynchronous listener events deterministically; this proves the selected execution path without entering global stage discovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rana Singh (ranadeepsingh)
added a commit
to ranadeepsingh/SynapseML
that referenced
this pull request
Aug 7, 2026
## Summary Use barrier execution only when VowpalWabbit training enables it and the prepared dataset has more than one partition. Add Spark-stage execution-path assertions for enabled multi-partition, disabled multi-partition, and enabled single-partition training. ## Prompting Intent Revalidate the current VowpalWabbitBaseLearner behavior, restore the minimal source-compatible rule proposed by the ancient PR, prove it through TDD and targeted VW validation, and keep the change isolated from LightGBM. ## Linked Sources - Original proposal: microsoft#1912 - Initial CI failure: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=229206650 - Repository review guidance: .github/skills/code-review/SKILL.md - Local validation guidance: .github/skills/synapseml-local-setup/SKILL.md ## Rationale Keep the public API and multi-partition synchronization behavior unchanged with a short-circuit partition-count guard. Observe Spark stage metadata rather than adding a test-only PipelineStage subclass, and use a marker job to drain asynchronous listener events deterministically; this proves the selected execution path without entering global stage discovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rana Singh (ranadeepsingh)
added a commit
that referenced
this pull request
Aug 7, 2026
## Summary Use barrier execution only when VowpalWabbit training enables it and the prepared dataset has more than one partition. Add Spark-stage execution-path assertions for enabled multi-partition, disabled multi-partition, and enabled single-partition training. ## Prompting Intent Revalidate the current VowpalWabbitBaseLearner behavior, restore the minimal source-compatible rule proposed by the ancient PR, prove it through TDD and targeted VW validation, and keep the change isolated from LightGBM. ## Linked Sources - Original proposal: #1912 - Initial CI failure: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=229206650 - Repository review guidance: .github/skills/code-review/SKILL.md - Local validation guidance: .github/skills/synapseml-local-setup/SKILL.md ## Rationale Keep the public API and multi-partition synchronization behavior unchanged with a short-circuit partition-count guard. Observe Spark stage metadata rather than adding a test-only PipelineStage subclass, and use a marker job to drain asynchronous listener events deterministically; this proves the selected execution path without entering global stage discovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
|
Superseded by #2592, which rebased this behavior onto current master, added targeted single- and multi-partition regression coverage, passed validation, and has now merged. Closing this historical PR to avoid duplicate work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
only require barrier mode if more than one partition is trained on