Skip to content

[Feat] Add FoldPythonScalarConstantsPass to fold pure Python scalar nodes before piecewise split#45

Merged
jiahy0825 merged 2 commits into
SandAI-org:mainfrom
Young-Leo:ly/fold-python-scalar-constants
Jul 17, 2026
Merged

[Feat] Add FoldPythonScalarConstantsPass to fold pure Python scalar nodes before piecewise split#45
jiahy0825 merged 2 commits into
SandAI-org:mainfrom
Young-Leo:ly/fold-python-scalar-constants

Conversation

@Young-Leo

Copy link
Copy Markdown
Contributor

What

Add FoldPythonScalarConstantsPass to the full-graph pass pipeline (runs before split_module). It evaluates an allowlist of pure scalar operators whose inputs are already plain Python scalars, replaces the FX node with the resulting literal, and removes the now-dead node.

Why

After a TensorifyScalarRestartAnalysis recapture, a tensor-derived SymFloat (e.g. the FP8 clamp bound -448.0) can come back as a concrete Python constant while the scalar arithmetic stays as an FX node (operator.neg(448.0)). When that value is reused across a piecewise boundary, split_module promotes the node to a subgraph output — and Inductor rejects a bare Python float as a compiled-graph output, crashing compilation.

torch has no native pass that folds Python-scalar FX nodes into literals.

  • torch._inductor.constant_folding only folds tensor constants (it explicitly skips nodes with no tensor input, and materializes results as get_attr buffers), and it runs inside each subgraph after split_module — too late for our case.
  • torch.fx.experimental.const_fold.split_const_subgraphs restructures the graph via its own split_module, conflicting with our piecewise splitting. So this small, pre-split scalar-literal fold fills a real gap rather than duplicating existing functionality.

Scope / Safety

  • Only an allowlist of pure scalar operators is evaluated.
  • Folds only when all inputs are already plain Python scalars; Tensor / SymInt / SymFloat / runtime-dependent values are never touched.
  • Arithmetic errors are caught and skipped; runs unconditionally on every (re)capture, so it also handles graphs that no longer contain item().

Tests

  • Unit tests: constant-chain folding, no-op on tensor/symbolic inputs, and no execution of non-allowlisted call_functions.
  • End-to-end regression reproducing the FP8 clamp-across-boundary crash.

Dynamo may recapture a tensor-derived SymFloat (e.g. the FP8 clamp bound
-448.0) as a plain Python constant while keeping the scalar arithmetic
(operator.neg(448.0)) as an FX node. When that value is reused across a
piecewise boundary, split_module promotes the node to a subgraph output,
and Inductor rejects a bare Python float as a compiled-graph output.

Add FoldPythonScalarConstantsPass to the full-graph pass pipeline (before
split_module). It evaluates only an allowlist of pure scalar operators
whose inputs are already plain Python scalars, replaces the FX node with
the resulting literal, then removes the dead node. Tensor / SymInt /
SymFloat / runtime-dependent values are never folded. The pass runs
unconditionally on every (re)capture, so it also handles graphs that no
longer contain item() after a TensorifyScalarRestartAnalysis restart.

Add unit tests for the pass plus an end-to-end regression reproducing the
FP8 clamp-across-boundary case.

@jiahy0825 jiahy0825 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiahy0825
jiahy0825 merged commit b01038e into SandAI-org:main Jul 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants