Remove Optim.jl interface + minor tidying up of src/optimisation/Optimisation #2708

penelopeysm · 2025-11-03T10:40:31Z

notes in comments.

Further refactoring will happen, but in separate PRs.

Closes #2635.

github-actions · 2025-11-03T10:43:39Z

Turing.jl documentation for PR #2708 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2708/

penelopeysm · 2025-11-03T10:41:29Z

test/ext/OptimInterface.jl

I checked and the base optimisation tests are a superset of these tests, so no need to incorporate any of these deleted tests into the main suite.

penelopeysm · 2025-11-03T10:42:07Z

src/optimisation/Optimisation.jl

-    function OptimLogDensity(
-        model::DynamicPPL.Model,
-        getlogdensity::Function,
-        vi::DynamicPPL.AbstractVarInfo;
-        adtype::ADTypes.AbstractADType=Turing.DEFAULT_ADTYPE,
-    )
-        ldf = DynamicPPL.LogDensityFunction(model, getlogdensity, vi; adtype=adtype)
-        return new{typeof(ldf)}(ldf)
-    end
-    function OptimLogDensity(
-        model::DynamicPPL.Model,
-        getlogdensity::Function;
-        adtype::ADTypes.AbstractADType=Turing.DEFAULT_ADTYPE,
-    )
-        # No varinfo
-        return OptimLogDensity(
-            model,
-            getlogdensity,
-            DynamicPPL.ldf_default_varinfo(model, getlogdensity);
-            adtype=adtype,
-        )
-    end


this is just duplicating LogDensityFunction code. I thought it simpler to just construct the LDF and then wrap it.

codecov · 2025-11-03T10:51:52Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.33%. Comparing base (b59947a) to head (9068a70).
⚠️ Report is 1 commits behind head on breaking.

Additional details and impacted files

@@             Coverage Diff              @@
##           breaking    #2708      +/-   ##
============================================
- Coverage     86.86%   86.33%   -0.54%     
============================================
  Files            22       21       -1     
  Lines          1447     1383      -64     
============================================
- Hits           1257     1194      -63     
+ Misses          190      189       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

yebai · 2025-11-03T12:15:38Z

HISTORY.md


+## Optimisation interface
+
+The Optim.jl interface has been removed (so you cannot call `Optim.optimize` directly on Turing models).


I suggest that we define our own optimize function on top of DynamicPPL.LogDensityFunction to avoid the breaking change. The optimize function provides a natural alternative to AbstractMCMC.sample.

I don't really agree:

What is the purpose of avoiding the breaking change? The functionality is already present in maximum_likelihood(model) or maximum_a_posteriori(model) so this is just duplicate functionality.

A Turing.optimize function would still be a breaking change because this is not the same as Optim.optimize.

Thanks, Penny, for clarifying.

My main point is not breaking change, instead optimise mirrors the Turing.sample API well, and provides a generic interface for optimisation, and VI algorithms. Sorry for the confusion. Though I agree we ought to remove the Optim interface and define our own optimize method.

Sure, in that case we could start by renaming estimate_mode to optimize, and exporting it. That would give us out of the box

optimize(model, MLE(); kwargs...) optimize(model, MAP(); kwargs...)

and then the VI arguments could be bundled into a struct that would be passed as the second argument.

penelopeysm · 2025-11-29T17:21:01Z

src/optimisation/Optimisation.jl


-    log_density = OptimLogDensity(model, getlogdensity, vi)
+    # Note that we don't need adtype here, because it's specified inside the
+    # OptimizationProblem
+    ldf = DynamicPPL.LogDensityFunction(model, getlogdensity, vi)
+    log_density = OptimLogDensity(ldf)

    prob = Optimization.OptimizationProblem(log_density, adtype, constraints)


I'm actually unsure if this is 'best practice', although I've kept it this way since that's how it was written prior to this PR. What this effectively does is to tell Optimization.jl to use adtype to differentiate through the LogDensityFunction. It seems to me that it would be more appropriate to construct an LDF with the appropriate adtype, and then tell OptimizationProblem to use the implementation of logdensity_and_gradient when it needs a gradient.

That does feel like a better approach. Looks like OptimizationFunction does support both ways of doing it, though I don't see a way to make use of logdensity_and_gradient also computing the log density: https://docs.sciml.ai/Optimization/v4.8/API/optimization_function/#optfunction

Hmmmm, but if it then wants something like Hessians (for, er, ... full Newton's method?) then we can't provide that, unless we rewrite OptimLogDensity to also cache hessian preparation and stuff. Maybe best to just let Optimization do its thing then.

That's very slightly disappointing, but I suppose also it's probably quite unlikely that this would be a performance pain point (compared to the actual cost of evaluating the gradient), so happy to drop it.

mhauru · 2025-12-01T14:39:42Z

src/optimisation/Optimisation.jl

    ModeEstimator

 An abstract type to mark whether mode estimation is to be done with maximum a posteriori
-(MAP) or maximum likelihood estimation (MLE). This is only needed for the Optim.jl interface.


I'm a bit confused by the comment that is being removed. Wouldn't we want these types to exist regardless? Or was the intention of the comment to say that we should move to using distinct functions for MLE and MAP and not have this type? I appreciate you didn't write the comment that is being removed, and for extra irony, I may have written it.

Ah yes, I was confused too. I actually started this PR by deleting them! 😄 then I realised we can't delete them. No big deal, this PR will fix the confusion...

(I think in general these types are good to have, I'm quite happy to have these kind of enums)

mhauru · 2025-12-01T16:53:26Z

src/optimisation/Optimisation.jl


-    log_density = OptimLogDensity(model, getlogdensity, vi)
+    # Note that we don't need adtype here, because it's specified inside the
+    # OptimizationProblem
+    ldf = DynamicPPL.LogDensityFunction(model, getlogdensity, vi)
+    log_density = OptimLogDensity(ldf)

    prob = Optimization.OptimizationProblem(log_density, adtype, constraints)


That does feel like a better approach. Looks like OptimizationFunction does support both ways of doing it, though I don't see a way to make use of logdensity_and_gradient also computing the log density: https://docs.sciml.ai/Optimization/v4.8/API/optimization_function/#optfunction

penelopeysm changed the base branch from main to breaking November 3, 2025 10:40

github-actions bot assigned penelopeysm Nov 3, 2025

penelopeysm force-pushed the py/no-optim branch from 6409018 to 02b1f48 Compare November 3, 2025 10:40

penelopeysm commented Nov 3, 2025

View reviewed changes

yebai reviewed Nov 3, 2025

View reviewed changes

penelopeysm force-pushed the py/no-optim branch 2 times, most recently from 33ecb77 to 6448372 Compare November 7, 2025 22:06

This was referenced Nov 8, 2025

Remove OptimizationOptimJL dep? #2712

Open

Update for DynamicPPL 0.39 #2715

Merged

penelopeysm added 6 commits November 29, 2025 17:13

Remove Optim.jl extension

858cda5

Simplify OptimLogDensity construction

c2ea09f

Changelog

a98ac14

Remove Optim.jl interface tests

4b53b09

Fix test

6e68ca4

fix a bug

9068a70

penelopeysm force-pushed the py/no-optim branch from fa46a89 to 9068a70 Compare November 29, 2025 17:18

penelopeysm commented Nov 29, 2025

View reviewed changes

penelopeysm requested a review from mhauru December 1, 2025 09:39

mhauru approved these changes Dec 1, 2025

View reviewed changes

penelopeysm merged commit 1d10d2d into breaking Dec 1, 2025
26 of 28 checks passed

penelopeysm deleted the py/no-optim branch December 1, 2025 18:04

penelopeysm mentioned this pull request Dec 1, 2025

Remove Optim.jl interface #2635

Closed


		## Optimisation interface

		The Optim.jl interface has been removed (so you cannot call `Optim.optimize` directly on Turing models).

Remove Optim.jl interface + minor tidying up of src/optimisation/Optimisation #2708

Remove Optim.jl interface + minor tidying up of src/optimisation/Optimisation #2708

Conversation

penelopeysm commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yebai Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

penelopeysm commented Nov 3, 2025 •

edited

Loading

codecov bot commented Nov 3, 2025 •

edited

Loading

yebai Nov 4, 2025 •

edited

Loading