[devmotion]

Adds rules for LogExpFunctions.logcosh, logabssinh, logabstanh, loglogistic, log1mlogistic, logitexp and logit1mexp, and makes the existing rules for logistic, log1psq, log1mexp, logexpm1, logaddexp and logsubexp more accurate. Fixes JuliaStats/LogExpFunctions.jl#61 (NaN derivatives of logaddexp(±Inf, ±Inf) in ForwardDiff).

New rules

function	derivative
logcosh(x)	tanh(x)
logabssinh(x)	coth(x)
logabstanh(x)	2 / sinh(2x)
loglogistic(x)	logistic(-x)
log1mlogistic(x)	-logistic(x)
logitexp(x)	-inv(expm1(x))
logit1mexp(x)	inv(expm1(x))

The rules for logcosh and logabssinh match the ChainRules rules in LogExpFunctions' ChainRulesCoreExt. For logabstanh, ChainRules uses inv(cosh(x) * sinh(x)); the equivalent 2 / sinh(2x) needs one transcendental call instead of two and is more accurate (max error vs. a BigFloat reference: 2.1 vs. 3.3 ulp for Float64, 1.8 vs. 3.0 ulp for Float32). Both forms agree at ±0, ±Inf and NaN.

Without these rules, ForwardDiff differentiates through the generic implementations, so Duals miss the specialized Float32/Float64 kernels (e.g. the polynomial kernel of logcosh). With the rules, the primal value of a Dual is identical (===) to the Float64 result, and the derivative comes from the closed form. See also JuliaStats/LogExpFunctions.jl#136, which fixed the accuracy of the generic logcosh path for Duals.

Improved rules

The rules are ported from JuliaStats/LogExpFunctions.jl#138, which makes the same changes to the ChainRules rules.

function	before	after
logistic(x)	z * (1 - z) with z = logistic(x)	e / (1 + e)^2 with e = exp(-abs(x))
log1psq(x)	2x / (1 + x^2)	same for abs(x) < 1, else 2 / (x + inv(x))
log1mexp(x)	-exp(x - log1mexp(x))	-inv(expm1(-x))
logexpm1(x)	exp(x - logexpm1(x))	-inv(expm1(-x))
logaddexp(x, y)	exp(x - logaddexp(x, y)), exp(y - logaddexp(x, y))	logistic(x - y), logistic(y - x)
logsubexp(x, y)	±exp(x - z), ∓exp(y - z) with z = logsubexp(x, y)	-inv(expm1(y - x)), -inv(expm1(x - y))

Max error vs. a BigFloat reference (Float64, |x| from 1e-300 to 700), before → after: logistic 8.5e15 → 1.5 ulp (cancellation in 1 - z), log1psq 9e15 → 1.0 ulp (overflow of x^2), log1mexp and logexpm1 502 → 1.0 ulp. logaddexp and logsubexp are limited by the rounding of x - y; all other rules are accurate to ≤ 1.5 ulp (Float32 and Float64). The new rules are also faster: per call, logaddexp 41 → 7.5 ns, logsubexp 25 → 10 ns, log1mexp/logexpm1 ~20 → 5 ns (log1psq pays ~0.4 ns for the branch).

Behaviour changes

• logaddexp(x, y) with x == y == ±Inf: partial derivatives (1/2, 1/2) instead of (NaN, NaN). As in LogExpFunctions.logaddexp, the NaN difference x - y is replaced with zero. This fixes ForwardDiff.derivative(t -> logaddexp(-Inf * t, -Inf * t), 1.0) (now -Inf) and logaddexp(Dual(-Inf, 0.0), Dual(-Inf, 0.0)) (now Dual(-Inf, 0.0)) without NaN-safe mode.
• logsubexp(x, x): partial derivatives (Inf, -Inf) instead of (-Inf, Inf), the limits as x approaches y from above (where logsubexp is differentiable). This includes x == y == -Inf (as in LogExpFunctions.logsubexp); x == y == Inf still gives NaN.
• The replacement of x - y with zero is restricted to infinite arguments: for finite x == y, x - y is already zero, and replacing it would drop higher-order derivatives in nested AD with number types whose == only compares primal values.

Changes

• New and updated rules in src/rules.jl.
• LogExpFunctions compat bumped from 0.3.2, 1 to 1: logcosh was added in 0.3.6, logabssinh in 0.3.24, logabstanh in 1.0.0, and loglogistic, log1mlogistic, logitexp and logit1mexp in 0.3.28.
• Tests:
  • logabssinh and logabstanh have a log singularity at 0, and logitexp and logit1mexp one at 0 with domain x < 0, so their finite-difference check points are shifted away from 0.
  • With LogExpFunctions ≥ 1, every unary rule has Float32 and Float64 methods, so the hasmethod guard around the finite-difference check is removed.
  • logistic, log1psq, log1mexp and logexpm1 are added to the accuracy tests from Avoid catastrophic cancellation in the derivatives of tanh and inverse trigonometric/hyperbolic functions #122 (rule evaluated in Float32/Float64 vs. in 256-bit BigFloat), at arguments where the previous rules fail: large |x| for logistic and log1psq, and small |x| for log1mexp and logexpm1.
  • New tests for Inf/NaN arguments of logaddexp, logsubexp, log1mexp and logexpm1, and for the accuracy of logaddexp and logsubexp (arguments of large magnitude and nearly equal arguments). These fail with the previous rules.

Note for #120: test/baseline.jl will need entries for the seven new rules.

🤖 Generated with Claude Code

[devmotion]

The Tracker downstream failure is pre-existing (also on master) and unrelated to this PR: Tracker's promote_rule throws a TypeError when promoting TrackedReal with non-Real types, which Julia 1.13's Test triggers. Fix: FluxML/Tracker.jl#175.