Skip to content

build(deps): make polars, dask and tqdm optional, drop dead dependencies - #1032

Open
olivier-lacroix wants to merge 1 commit into
PyPSA:masterfrom
olivier-lacroix:dep-trim
Open

olivier-lacroix wants to merge 1 commit into
PyPSA:masterfrom
olivier-lacroix:dep-trim

Conversation

@olivier-lacroix

@olivier-lacroix olivier-lacroix commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Linopy currently requires some unused dependencies, and some which are used only in a select number of methods. This PR removes unused dependencies, and make some others optional

Note

The following content was generated by AI.

What

The declared dependency list had 11 entries. Instrumenting the test suite showed which ones the code actually reaches.

Dependency Measured use Outcome
toolz never imported; a hard requirement of dask removed, arrives with the dask extra
numexpr 0 calls across 8054 tests removed
dask never imported; reached only through xarray's .chunk() linopy[dask]
tqdm 6 progress bars in the LP writer linopy[progress]
deprecation one decorator; the other 30 deprecation sites call warnings.warn removed
polars to_polars() and the LP writer linopy[polars]
bottleneck 286 nanmax, 284 nanmin, 16 push calls stays required

Core dependencies: numpy, scipy, xarray, pandas, packaging, bottleneck. pandas is declared explicitly, since 18 modules import it and it previously arrived only through xarray.

Why bottleneck stays

xarray's duck_array_ops._push imports bottleneck directly, with no numpy fallback. Variable.ffill and Variable.bfill go through it, so a core-only install raised ModuleNotFoundError from both. A core-only venv confirmed it, and the no-extras CI job now calls ffill and bfill so this cannot regress silently.

The nanmin/nanmax calls come from xarray's nanops behind .min()/.max() on NaN-carrying label and coefficient arrays, as used by Constraints.coefficientrange and the objective range.

numexpr stays out: linopy never calls pd.eval or Dataset.query, and xarray's accel extra does not list it.

How polars became optional

polars is loaded through a placeholder that raises ImportError with an install hint on first use. import linopy, model building, direct-API solves, MPS export and Variable.ffill all work without it. The default Model.solve() writes an LP file and still needs polars; to_lp_file raises a message naming linopy[polars] and pointing at io_api="direct".

get_printers_scalar drops polars for numpy, which removes polars from the direct-API solve path shared by the Highs, Gurobi, Cplex and Xpress backends.

New extras: polars, dask, progress. The dev and docs extras pull them in through a self-reference, so the extra list lives in one place.

Tests

  • New tests in test/test_io.py cover the tqdm fallback, the LP guard with polars hidden from sys.modules, the placeholder, and the dunder probe that pytest's doctest collection makes.
  • A new CI job installs the wheel with no extras and smoke-tests the core path.
  • Generated LP files and printed variable and constraint names are byte-identical to master.
How the usage was measured, and the full suite results

A pytest plugin wrapped every public callable of bottleneck and numexpr before collection and counted calls, recording the first-call stack for each. Running it over the whole suite gave 586 bottleneck calls and 0 numexpr calls.

Stack traces pointed at:

  • xarray/computation/nanops.py nanmin/nanmax, reached from constraints.py:2627 coefficientrange
  • xarray/core/duck_array_ops.py:892 _push, reached from xarray/core/dataarray.py:3858 bfill

@codspeed

codspeed Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks1


Comparing olivier-lacroix:dep-trim (5194de8) with master (2d6aaf9)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

The declared dependency list had 11 entries. Instrumenting the test suite with
a plugin that wraps every public callable of bottleneck and numexpr showed
which ones the code actually reaches.

Removed:

- toolz, never imported. It is a hard requirement of dask, so it arrives with
  the new dask extra.
- numexpr, 0 calls across 8054 tests. linopy never calls pd.eval or
  Dataset.query, and xarray's accel extra does not list it.
- deprecation, which backed a single decorator. The other 30 deprecation sites
  in the codebase call warnings.warn directly.

Moved behind extras:

- polars backs to_polars() and the LP file writer, so it becomes
  linopy[polars].
- tqdm backs the LP writer's progress bars, so it becomes linopy[progress].
- dask is never imported by linopy, only reached through xarray's .chunk(),
  so it becomes linopy[dask].

Kept:

- bottleneck, with 286 nanmax, 284 nanmin and 16 push calls measured. xarray's
  duck_array_ops._push imports bottleneck directly with no numpy fallback, so
  Variable.ffill and Variable.bfill raised ModuleNotFoundError on a core-only
  install. The no-extras CI job now calls both, so a packaging change that
  drops bottleneck fails the build.

Added: pandas, which 18 modules import and which previously arrived only
through xarray.

Core dependencies are now numpy, scipy, xarray, pandas, packaging and
bottleneck.

polars is loaded through a placeholder that raises ImportError with an install
hint on first use, so import linopy, model building, direct-API solves, MPS
export and Variable.ffill all work without it. The default Model.solve()
writes an LP file and still needs polars; to_lp_file raises a message naming
linopy[polars] and pointing at io_api="direct".

get_printers_scalar drops polars for numpy, which removes polars from the
direct-API solve path shared by the Highs, Gurobi, Cplex and Xpress backends.
Generated LP files and printed variable and constraint names are byte-identical
to master.

The dev and docs extras pull the new extras in through a self-reference, so
the extra list lives in one place. New tests in test/test_io.py cover the tqdm
fallback, the LP guard with polars hidden from sys.modules, the placeholder,
and the dunder probe that pytest's doctest collection makes.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant