Skip to content

Take higher-order derivative expressions from DiffRules instead of evaluating ChainRules rules symbolically - #114

Merged
tansongchen merged 2 commits into
mainfrom
dmw/diffrules-higher-order
Nov 11, 2026
Merged

tansongchen merged 2 commits into
mainfrom
dmw/diffrules-higher-order

Conversation

@devmotion

@devmotion devmotion commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Fixes #113.

Problem

To raise unary functions to higher orders, get_term_raiser evaluates ChainRules' frule on a Symbolics variable during precompilation. The resulting expression is then evaluated on Taylor series. This relies on two properties that ChainRules doesn't promise, since its rules are only meant to be evaluated on numbers:

  1. The rule must not branch on values. Avoid catastrophic cancellation in tanh and inverse trig/hyperbolic derivatives ChainRules.jl#864 (ChainRules 1.74.11) made the tanh rule abs(real(x)) < 1 ? 1 - Ω^2 : sech(x)^2, so TaylorDiff fails to precompile with ChainRules ≥ 1.74.11. A branch on the value is fine for numbers. A Symbolics variable can't be compared, though, so precompilation throws.
  2. The rule must equal the derivative as a function, not only numerically, and its algebraic shape decides the raising strategy. A rewrite of a rule in ChainRules can therefore silently change TaylorDiff's accuracy or performance.

Changes

  • get_term_raiser takes the derivative from DiffRules.diffrule(M, f, :z). DiffRules' expressions are value-free by design, and Symbolics' own derivatives are built from them.
    • It picks raiseinv when the expression is inv(g), 1 / g or a negation of those, matched with MacroTools.@capture; otherwise raise.
    • The raising strategy is the same as before for all 23 Base functions.
    • abs (sign(z), as before) and sign (zero; DiffRules has no rule) are defined explicitly.
  • Symbolics and SymbolicUtils are dropped as dependencies, along with the pkgversion(Symbolics) branches. using TaylorDiff goes from 1.76 s to 0.38 s (warm, precompiled).
  • TaylorDiffSFExt imports the names its generated methods refer to by name. It was broken before: every SpecialFunctions derivative threw UndefVarError: value not defined in TaylorDiffSFExt.
  • First-order derivatives still call frule on numbers, unchanged.

This requires DiffRules 1.17 (JuliaDiff/DiffRules.jl#122), which is now released and registered. DiffRules 1.16 uses sqrt(x^2 - 1) for acosh, which would be less accurate near 1 than the sqrt(x - 1) * sqrt(x + 1) that TaylorDiff got from ChainRules 1.74.10. DiffRules 1.17 also makes higher-order tanh, atanh, acoth etc. accurate in the cases where they weren't before.

Verification

I compared derivative(f, x, Val(k)) for k = 1:4 against TaylorDiff 0.3.5 with ChainRules 1.74.10, for all 28 functions at 2–3 points each (including near-singular ones), in Float64 and Float32, with BigFloat as the reference:

  • Nothing that worked before errors now, and no result type changes.
  • No result is less accurate. Near 1, atanh/acoth improve from 10–28 ulps to about 1 ulp.
  • These now work: all SpecialFunctions functions (Float64 and Float32) and deg2rad/rad2deg in Float32.

New tests:

  • Each expression agrees with frule at in-domain points.
  • The raising strategy for each function.
  • Float32 is preserved for higher-order derivatives.

SpecialFunctions is added to the test dependencies to cover the extension.

A known issue remains, unchanged by this PR: acos, acosh, acsc, acsch, asec, asech, asin and asinh still error for Float32 inputs at orders ≥ 2. The cause is sqrt(t::TaylorScalar) = t^0.5 (src/primitive.jl), which promotes to Float64. I can open a separate PR for that.

Version bumped to 0.3.6. A patch release would make TaylorDiff load again with current ChainRules.

CI notes

  • Format Check also fails on main (since April). The latest JuliaFormatter reformats code that predates this PR (examples/, src/scalar.jl, process in src/utils.jl, older tests in test/primitive.jl). This PR only fixes the formatting of the lines it adds. The rest can go in a separate PR.
  • "Zygote-over-TaylorDiff on same variable" (test/downstream.jl:34, matrix exp gradient) fails locally on both main and this PR with ChainRules 1.74.10, Zygote 0.7.16 and DifferentiationInterface 0.7.22, so the failure predates this PR.

🤖 Generated with Claude Code

devmotion and others added 2 commits November 10, 2026 02:29
`get_term_raiser` evaluated ChainRules' `frule` on a Symbolics variable,
which breaks as soon as a rule branches on values (#113, ChainRules
1.74.11). DiffRules' expressions are value-free by design, so take the
derivative from `DiffRules.diffrule` and drop the Symbolics and
SymbolicUtils dependencies. Also import the names used by the code
generated in `TaylorDiffSFExt`, which was broken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Format the new Float32 testset as JuliaFormatter (sciml) expects
- Don't link to the undocumented `raise`/`raiseinv` in the `get_term_raiser`
  docstring; it is rendered by `@autodocs` and broke the docs build
- Don't import `raise`/`raiseinv` in TaylorDiffSFExt; the generated code
  interpolates the function objects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SebastianM-C

Copy link
Copy Markdown

Thanks for this! This solves the precompilation errors with TaylorDiff

@tansongchen

Copy link
Copy Markdown
Member

Thanks this makes sense

@tansongchen
tansongchen merged commit 41d338b into main Nov 11, 2026
1 of 4 checks passed
@tansongchen
tansongchen deleted the dmw/diffrules-higher-order branch November 11, 2026 12:14
@devmotion

Copy link
Copy Markdown
Member Author

Thank you! Could you tag a release with this change @tansongchen? I already updated the version number in this PR.

@tansongchen

Copy link
Copy Markdown
Member

just did!

@devmotion

Copy link
Copy Markdown
Member Author

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fails to precompile with ChainRules 1.74.11: non-boolean Symbolics.Num in the tanh rule

3 participants