fix(solvers): dispose the solver model before its env on close by FabianHofmann · Pull Request #899 · PyPSA/linopy

FabianHofmann · 2026-08-18T14:17:02Z

The current CI on master is flaky with COPT, let's test if it has to do with the potential issue raised and fixed here

Note

The following content was generated by AI.

Changes proposed in this Pull Request

Solver.close() closed the environment before dropping the model that lives in it:

if self._env_stack is not None:
    self._env_stack.close()   # env goes away
self.env = None
self.solver_model = None      # native model destructor runs, env already gone

Solvers that register their model on the environment's ExitStack — Gurobi (since #826), Xpress, Mosek — get the right LIFO ordering and were unaffected. HiGHS, SCIP, COPT and MindOpt only dereference the model, so its native destructor ran against a freed environment. Because close() is also called from __del__, the crash lands in whatever code happened to trigger the collection, which makes it look unrelated and intermittent.

This is what CI has been hitting on master:

Windows fatal exception: access violation, faulting frame solvers.py close() at self.solver_model = None, reached from __del__ during a garbage collection pass inside unrelated xarray alignment code.
Fatal Python error: Aborted inside COPT._run_file at env_.createModel() — a later COPT call aborting on state a previous freed model corrupted.

Two changes:

Solver.close() drops solver_model first, then closes the environment stack. For solvers whose model sits on the stack this is a no-op (the stack still holds the reference and disposes it first); for the others the model is now released while its environment is still alive.
COPT._run_file no longer closes its environment in a finally while returning the model built in it. The environment is registered on the solver's ExitStack, so it is released by Solver.close() and model.solver_model stays usable after model.solve("copt") — matching Gurobi's behaviour.

Verification

Both crashes are macOS/Windows-only and were not reproducible on Linux, so this fix is validated by CI rather than locally. Locally (Linux, Python 3.13) the full suite passes: 2980 passed, 33 skipped, with test/remote and Xpress excluded — Xpress is deselected because of an unrelated local install problem (NameError: name 'xpress' is not defined), and test/remote needs OETC credentials. mypy reports no new errors, and ruff check / ruff format are clean.

test_copt_env_persists_after_solve pins the COPT invariant. It is skipped locally (no valid COPT licence on this machine) and runs on the CI hosts where COPT is licensed.

Crash traceback (Windows, master and PR #717)

Windows fatal exception: access violation

Current thread 0x00001430 (most recent call first):
  Garbage-collecting
  File "D:\a\linopy\linopy\linopy\solvers.py", line 1130 in close
  File "D:\a\linopy\linopy\linopy\solvers.py", line 1135 in __del__
  File "...\xarray\core\indexes.py", line 1798 in get_unique
  File "...\xarray\structure\alignment.py", line 247 in _collect_indexes
  File "...\xarray\structure\merge.py", line 757 in merge_core
  ...

Crash traceback (macOS, master run 32129308681)

Fatal Python error: Aborted

Current thread 0x00000001fbee5d80 (most recent call first):
  File "/Users/runner/work/linopy/linopy/linopy/solvers.py", line 3942 in _run_file
  File "/Users/runner/work/linopy/linopy/linopy/solvers.py", line 861 in solve
  File "/Users/runner/work/linopy/linopy/linopy/model.py", line 2136 in solve
  File "/Users/runner/work/linopy/linopy/test/test_optimization.py", line 536 in test_default_settings_small_slices

Checklist

AI-generated content is marked (see AGENTS.md).
Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Unit tests for new features were added (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the MIT license.

Solver.close() closed the env ExitStack before dropping solver_model, so the native model was collected against freed memory — a Fatal Python error or Windows access violation from an unrelated GC pass. COPT also closed its env in a finally while returning the model built in it; the env now lives on the solver's ExitStack.

codspeed-hq · 2026-08-18T14:20:16Z

Merging this PR will degrade performance by 3.51%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 4 regressed benchmarks
✅ 169 untouched benchmarks
⏩ 175 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Memory	`test_to_lp[milp-n=50]`	2 MB	2.7 MB	-24.39%
❌	Memory	`test_to_lp[nodal_balance-severity=50]`	2.8 MB	3.4 MB	-18.18%
❌	Memory	`test_to_lp[merge_balance-severity=0]`	2.6 MB	3.2 MB	-17.89%
❌	Memory	`test_to_lp[rolling-severity=0]`	2.7 MB	3.1 MB	-12.56%
⚡	Memory	`test_to_lp[rolling-severity=50]`	429.7 MB	305.6 MB	+40.61%
⚡	Memory	`test_to_lp[masked-n=100]`	2.7 MB	2.1 MB	+29.24%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing fix/solver-close-ordering (ee50956) with master (09c34dd)}

175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2026-08-19T06:34:46Z

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08063 (1.12)   │   13.33 (1.00) 
 (v1)        0.07184 (1.0)   │    13.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0842 (1.09)   │   11.09 (1.00) 
 (v1)       0.07718 (1.0)   │    11.09 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03334 (1.10)   │   15.17 (1.0) 
 (v1)        0.03045 (1.0)   │   15.17 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04613 (1.07)   │   44.93 (1.0) 
 (v1)        0.04307 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03666 (1.08)   │   11.51 (1.0) 
 (v1)        0.03386 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08922 (1.06)   │   24.23 (1.10) 
 (v1)        0.08449 (1.0)   │    22.05 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1006 (1.08)   │   15.64 (1.00) 
 (v1)       0.09342 (1.0)   │    15.64 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02186 (1.07)   │   752.15 (1.10) 
 (v1)        0.02045 (1.0)   │    685.11 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02167 (1.08)   │   3.09 (1.34) 
 (v1)        0.02015 (1.0)   │    2.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05889 (1.23)   │   126.16 (1.44) 
 (v1)        0.04795 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05779 (1.23)   │   126.16 (1.44) 
 (v1)        0.04697 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05855 (1.24)   │   126.16 (1.44) 
 (v1)        0.04711 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04756 (1.01)   │    714.28 (1.0) 
 (v1)        0.04693 (1.0)   │   785.32 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04625 (1.11)   │   4.40 (1.34) 
 (v1)         0.0418 (1.0)   │    3.27 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.3568 (1.04)   │   702.97 (1.10) 
 (v1)        0.3415 (1.0)   │    639.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.3721 (1.04)   │   18.34 (1.0) 
 (v1)        0.3564 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.3697 (1.04)   │   9.54 (1.0) 
 (v1)        0.3548 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.06747 (1.09)   │   3.64 (1.14) 
 (v1)         0.0621 (1.0)   │    3.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06776 (1.07)   │   216.15 (1.10) 
 (v1)        0.06312 (1.0)   │    195.76 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03306 (1.08)   │   935.30 (1.0) 
 (v1)        0.03061 (1.0)   │   935.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03433 (1.09)   │   9.66 (1.0) 
 (v1)         0.0315 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03389 (1.09)   │   5.32 (1.0) 
 (v1)        0.03117 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1678 (1.07)   │   891.72 (1.07) 
 (v1)        0.1566 (1.0)   │    836.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1638 (1.06)   │   11.02 (1.03) 
 (v1)        0.1548 (1.0)   │    10.67 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0457 (1.07)   │   147.66 (1.06) 
 (v1)       0.04285 (1.0)   │    139.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04533 (1.06)   │   2.57 (1.09) 
 (v1)        0.04261 (1.0)   │    2.35 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03392 (1.09)   │   696.72 (1.14) 
 (v1)        0.03126 (1.0)   │    611.13 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.07763 (1.09)   │   137.97 (1.0) 
 (v1)         0.0715 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05213 (1.08)   │   69.22 (1.0) 
 (v1)        0.04813 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03881 (1.11)   │   377.39 (1.04) 
 (v1)        0.03497 (1.0)   │    363.14 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03824 (1.10)   │   3.10 (1.41) 
 (v1)        0.03463 (1.0)   │    2.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04408 (1.08)   │   28.94 (1.61) 
 (v1)        0.04094 (1.0)   │    17.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05485 (1.10)   │   37.95 (1.43) 
 (v1)        0.04989 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.08748 (1.0)   │    397.39 (1.0) 
 (v1)       0.09167 (1.05)   │   414.56 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.09299 (1.0)   │    9.62 (1.0) 
 (v1)       0.0973 (1.05)   │   9.90 (1.03)

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}

FBumann · 2026-08-25T08:49:39Z

@FabianHofmann Lets land this to get the other PRs green again!

…is a list coptpy returns a coptcore.VarArray, so the isinstance check could never pass where COPT is installed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

A Solver is only ever reachable in a Model/Solver reference cycle, so it is reclaimed by the cyclic collector and never by refcount. Its __del__ therefore ran mid-collection, tearing down native solver handles at an arbitrary point in an unrelated call stack — the interpreter aborted in coptpy.Envr(), cplex.Cplex(), GLPK's result parsing and inside xarray alignment, in a different CI job on every run. close() is unchanged and is still called when a new solve replaces the solver and when model.solver is reassigned. Solvers that are never closed are left to the vendor wrappers, which dispose in their own finalizers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The file interface built a cplex.Cplex() per solve, handed it back as solver_model and never ended it, so every solve left a live CPLEX environment for the garbage collector. Collecting one re-entered the CPLEX library while another environment was being constructed, which aborted the interpreter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

This reverts commit c420696.

Keeping the environment on the solver's ExitStack leaked it: with no finalizer, a solver that is merely dropped — every solve in the test suite builds a fresh Model and drops it — never runs close(), so the callback never fires. Measured on the file-based path: 20 solves created 20 COPT environments and closed none, against 20/20 on master. Close per solve as before, but do not hand back a solver model whose environment has just been closed; model.solver_model is None after a file-based COPT solve rather than a handle into freed memory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

MindOpt disposes both the model and the environment before returning, so the handle it handed back as solver_model pointed at freed memory — the same defect as COPT. Knitro already does this correctly: it extracts what it needs into a plain dataclass before freeing the native context. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Restore an unrelated blank line, shorten the close() docstring and drop the typing import the spy no longer needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Removing Solver.__del__ withdrew a guarantee announced in 0.8.0, so say so where it was promised: Model.solve still claimed the solver is released on garbage collection, and the note sat under bug fixes rather than breaking changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

FBumann · 2026-08-25T11:47:51Z

@FabianHofmann This is quite tidious.

Maybe making Solver.model a weakref could fix it...

FBumann · 2026-08-27T07:22:13Z

@FabianHofmann This PR now fixes the bug with a small tradeoff.
Its yours to judge if thats an issue.

FabianHofmann · 2026-08-27T12:17:03Z

we can live with that

docs: condense the release note for the solver close fix

d57a462

FBumann mentioned this pull request Aug 18, 2026

fix: _as_index() calls to_index() for DataArrays #897

Merged

5 tasks

FabianHofmann added 2 commits August 19, 2026 08:25

Merge branch 'master' into fix/solver-close-ordering

d728a6e

Merge branch 'master' into fix/solver-close-ordering

f3a4405

FabianHofmann force-pushed the fix/solver-close-ordering branch from d97a8d1 to f3a4405 Compare August 19, 2026 06:31

FBumann and others added 9 commits August 25, 2026 10:51

test(solvers): assert the COPT model is readable, not that getVars() …

a8d1253

…is a list coptpy returns a coptcore.VarArray, so the isinstance check could never pass where COPT is installed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Merge branch 'master' into fix/solver-close-ordering

64f187b

Revert "fix(cplex): own the Cplex handle and end it on close"

34f03db

This reverts commit c420696.

chore: trim the diff to the fix

a7be805

Restore an unrelated blank line, shorten the close() docstring and drop the typing import the spy no longer needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

FBumann mentioned this pull request Aug 26, 2026

fix(solver): a gurobi solver releases its model and licence however it is dropped fluxopt/lpspec#1292

Merged

docs: tighten the solver-release release notes

ee50956

FabianHofmann merged commit 570ff2e into master Aug 27, 2026
9 of 11 checks passed

FabianHofmann deleted the fix/solver-close-ordering branch August 27, 2026 12:17

Navigation Menu

Uh oh!

fix(solvers): dispose the solver model before its env on close - #899

fix(solvers): dispose the solver model before its env on close#899
FabianHofmann merged 14 commits into
masterfrom
fix/solver-close-ordering

FabianHofmann commented Aug 18, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 19, 2026 •

edited

Loading

Uh oh!

FBumann commented Aug 25, 2026

Uh oh!

FBumann commented Aug 25, 2026

Uh oh!

FBumann commented Aug 27, 2026

Uh oh!

FabianHofmann commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

FabianHofmann commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request

Verification

Checklist

Uh oh!

codspeed-hq Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 3.51%

Performance Changes

Footnotes

Uh oh!

github-actions Bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build cost — v1 vs legacy

Uh oh!

FBumann commented Aug 25, 2026

Uh oh!

FBumann commented Aug 25, 2026

Uh oh!

FBumann commented Aug 27, 2026

Uh oh!

FabianHofmann commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

FabianHofmann commented Aug 18, 2026 •

edited

Loading

codspeed-hq Bot commented Aug 18, 2026 •

edited

Loading

github-actions Bot commented Aug 19, 2026 •

edited

Loading