[Feat] Add FoldPythonScalarConstantsPass to fold pure Python scalar nodes before piecewise split#45
Merged
jiahy0825 merged 2 commits intoJul 17, 2026
Conversation
Dynamo may recapture a tensor-derived SymFloat (e.g. the FP8 clamp bound -448.0) as a plain Python constant while keeping the scalar arithmetic (operator.neg(448.0)) as an FX node. When that value is reused across a piecewise boundary, split_module promotes the node to a subgraph output, and Inductor rejects a bare Python float as a compiled-graph output. Add FoldPythonScalarConstantsPass to the full-graph pass pipeline (before split_module). It evaluates only an allowlist of pure scalar operators whose inputs are already plain Python scalars, replaces the FX node with the resulting literal, then removes the dead node. Tensor / SymInt / SymFloat / runtime-dependent values are never folded. The pass runs unconditionally on every (re)capture, so it also handles graphs that no longer contain item() after a TensorifyScalarRestartAnalysis restart. Add unit tests for the pass plus an end-to-end regression reproducing the FP8 clamp-across-boundary case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add
FoldPythonScalarConstantsPassto the full-graph pass pipeline (runs beforesplit_module). It evaluates an allowlist of pure scalar operators whose inputs are already plain Python scalars, replaces the FX node with the resulting literal, and removes the now-dead node.Why
After a
TensorifyScalarRestartAnalysisrecapture, a tensor-derivedSymFloat(e.g. the FP8 clamp bound-448.0) can come back as a concrete Python constant while the scalar arithmetic stays as an FX node (operator.neg(448.0)). When that value is reused across a piecewise boundary,split_modulepromotes the node to a subgraph output — and Inductor rejects a bare Python float as a compiled-graph output, crashing compilation.torch has no native pass that folds Python-scalar FX nodes into literals.
torch._inductor.constant_foldingonly folds tensor constants (it explicitly skips nodes with no tensor input, and materializes results asget_attrbuffers), and it runs inside each subgraph aftersplit_module— too late for our case.torch.fx.experimental.const_fold.split_const_subgraphsrestructures the graph via its ownsplit_module, conflicting with our piecewise splitting. So this small, pre-split scalar-literal fold fills a real gap rather than duplicating existing functionality.Scope / Safety
item().Tests