TYP: Enhance static typing #3771

n-takumasa · 2025-10-24T05:25:06Z

Fixes # NA

Summary/Motivation:

To provide better autocompletion in vscode+pylance.
Complex dynamic patterns (e.g. Var() can return ScalarVar or IndexedVar) are out of scope as they are difficult to model accurately.

Changes proposed in this PR:

TYP: Use typing.overload during TYPE_CHECKING
TYP: Model.__new__ should return an instance of the subclass
TYP: better typing for SolverFactorys and pyomo.future.solver_factory
TYP: reorder @overloads, overlapped overloads never be used

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…ory`

jsiirola

Thanks for this! I have several questions (see below). Some are less about this PR specifically, and more about generally how we want to better support typing in Pyomo. It would be useful to talk through that sometime in a Tuesday Dev Call.

jsiirola · 2025-11-15T17:39:47Z

pyomo/common/pyomo_typing.py

+if typing.TYPE_CHECKING:
+    from typing import overload as overload
+else:


Why do you need to disable the overload?

A function calling typing.overload is not recognized as an overload by the type checker.
We need to replace it with typing.overload during type checking.

OK. I went digging and it turns out that logic is primarily needed by a dependent project -- but only for Python versions through 3.10. I think we should actually document that better in the code, with something like:

if sys.version_info[:2] <= (3, 10) and not TYPE_CHECKING: def overload(func: typing.Callable): """Wrap typing.overload that remembers the overloaded signatures This provides a custom implementation of typing.overload that remembers the overloaded signatures so that they are available for runtime inspection (backporting `get_overloads` from Python 3.11+). """ _overloads.setdefault(_get_fullqual_name(func), []).append(func) return typing.overload(func) def get_overloads_for(func: typing.Callable): return _overloads.get(_get_fullqual_name(func), []) else: from typing import overload, get_overloads as get_overloads_for

typing_extensions might be helpful.

jsiirola · 2025-11-16T18:36:23Z

pyomo/core/base/PyomoModel.py



+# NOTE: Python 3.11+ use `typing.Self`
+ModelT = TypeVar("ModelT", bound="Model")


We haven't really settled on a typing convention for naming types, but appending T seems confusing.

would standardizing on appendint Type be more clear?

should local TypeVar objects be private by default (i.e., _modelType here)?

_ModelType is better.

pyomo/core/base/block.py

jsiirola · 2025-11-16T18:39:44Z

pyomo/future.py

        for ver, cls in versions.items():
            if cls._cls is _environ.SolverFactory._cls:
-                solver_factory._active_version = ver
+                solver_factory._active_version = ver  # type: ignore


We shouldn't need these annotations, should we?

Because we cannot add attributes to FunctionType.

You can't annotate the function, plus the attribute whereit is first instantiated at the bottom of the file?

def solver_factory(version: int | None = None) -> int: # ... solver_factory._active_version: int = solver_factory()

It is allowed at runtime, but it's a type violation.
ref: microsoft/pyright#8838

pyomo/opt/base/solvers.py

codecov · 2025-11-17T18:50:24Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.42%. Comparing base (a079b25) to head (b4fcea0).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3771   +/-   ##
=======================================
  Coverage   89.41%   89.42%           
=======================================
  Files         909      909           
  Lines      105579   105589   +10     
=======================================
+ Hits        94407    94421   +14     
+ Misses      11172    11168    -4

Flag	Coverage Δ
builders	`29.11% <35.48%> (-0.01%)`	⬇️
default	`86.04% <51.61%> (?)`
expensive	`35.77% <48.38%> (?)`
linux	`86.73% <100.00%> (-2.46%)`	⬇️
linux_other	`86.73% <100.00%> (+<0.01%)`	⬆️
osx	`82.88% <100.00%> (+<0.01%)`	⬆️
win	`84.99% <100.00%> (+0.01%)`	⬆️
win_other	`84.99% <100.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jsiirola · 2025-12-11T19:58:37Z

We were talking though this PR on the dev call this week. I think there are 3 changes that need to be made to get it merged:

rename modelT to ModelType
rework the TYPE_CHECKING test, possibly using the suggestion from the review
refactor the solver_factory function to promote the state to a module-level variable so that it can be statically-typed in a way that makes pyright happy.

Co-Authored-By: John Siirola <356359+jsiirola@users.noreply.github.com>

n-takumasa · 2025-12-12T00:52:57Z

Almost done.

rename modelT to ModelType

Is this intended to be public?
For now, I committed it as private with _ModelType.

jsiirola

Some minor comments. The biggest is that I would prefer to see the changes to pyomo.future.solver_factory() be reverted with the exception of promoting the _active_version attribute on the function to a _active_solver_factory_version module attribute. The changes add an unnecessary (admittedly slight) overhead and break some of the documented functionality.

jsiirola · 2025-12-12T18:22:51Z

pyomo/future.py

+solver_factory_v1: _SolverFactoryClassV1 = _solvers.LegacySolverFactory
+solver_factory_v2: _SolverFactoryClassV2 = _appsi.SolverFactory
+solver_factory_v3: _SolverFactoryClassV3 = _contrib.SolverFactory


We don't want to declare these at the module scope: if we do, the module __getattr__ won't be called, which was how we were implementing the Python idiom:

from pyomo.__future__ import solver_factory_v3

as an alias for

from pyomo.__future__ import solver_factory solver_factory(3)

My bad, it's regression.
I didn't realize that importing changes the state.

jsiirola · 2025-12-12T18:26:07Z

pyomo/future.py

+solver_factory_v2: _SolverFactoryClassV2 = _appsi.SolverFactory
+solver_factory_v3: _SolverFactoryClassV3 = _contrib.SolverFactory
+
+_versions = {1: solver_factory_v1, 2: solver_factory_v2, 3: solver_factory_v3}


Future is intended to manage all "future compatibility changes", so we should use a more specific name than versions here - there will eventually be several "versions" that this module will manage.

I would prefer not promoting this dict to the module scope: solver_factory will probably never be called more than once, so the performance gain of preserving the dict isn't worth the memory.

jsiirola · 2025-12-12T18:28:11Z

pyomo/future.py

+_SolverFactoryClassV1 = _solvers.SolverFactoryClass
+_SolverFactoryClassV2 = _appsi.SolverFactoryClass
+_SolverFactoryClassV3 = _contrib.SolverFactoryClass


I would prefer not caching these at the module scope.

… variable" This reverts commit b5a45cb.

jsiirola

This is super close. If we can delete the 3 lines indicated, then I think this is good to go.

jsiirola · 2025-12-12T21:55:24Z

pyomo/future.py

+                _active_solver_factory_version = ver
                break
-        return solver_factory._active_version
+        return _active_solver_factory_version


[This is my fault] As I am thinking through this, I think this is actually a bug and this should be reworked. Right now, everything is "fine" because we are guaranteed that the first time this function is called, it will be called with version=None (that is hard-coded below and will happen on import). BUT that seems fragile. I think I would propose combining this logic with the if version is None below:

if version is None: if "_active_solver_factory_version" not in globals(): for ver, cls in versions.items(): if cls._cls is _environ.SolverFactory._cls: _active_solver_factory_version = ver break return _active_solver_factory_version

@n-takumasa: totally up to you if you want to make this change in this PR or leave it to me to get back to later

I agree because if we delete _active_solver_factory_version = solver_factory() and call solver_factory(int), it will break.

pyomo/future.py

Co-Authored-By: John Siirola <356359+jsiirola@users.noreply.github.com>

n-takumasa added 6 commits October 22, 2025 14:22

TYP: Use typing.overload during TYPE_CHECKING

4389d6d

TYP: Model.__new__ should return an instance of the subclass

4765c4a

TYP: better typing for SolverFactorys and `pyomo.future.solver_fact…

15528c3

…ory`

Merge branch 'main' into typing

360064e

Merge branch 'Pyomo:main' into typing

4be9b33

TYP: reorder @overloads, overlapped overloads never be used

58ec35d

blnicho requested review from jsiirola and michaelbynum October 28, 2025 19:09

blnicho added this to Pyomo 6.10 Nov 11, 2025

github-project-automation bot moved this to Todo in Pyomo 6.10 Nov 11, 2025

jsiirola reviewed Nov 16, 2025

View reviewed changes

jsiirola added the AT: PRE-TEST INSPECTED label Nov 17, 2025

pyomo-autotest removed the AT: PRE-TEST INSPECTED label Nov 17, 2025

Merge branch 'main' into typing

fb6859f

jsiirola added the AT: PRE-TEST INSPECTED label Nov 17, 2025

pyomo-autotest removed the AT: PRE-TEST INSPECTED label Nov 17, 2025

pyomo-autotest added the AT: STALE label Dec 7, 2025

Merge branch 'Pyomo:main' into typing

19de89a

pyomo-autotest removed the AT: STALE label Dec 11, 2025

n-takumasa and others added 4 commits December 12, 2025 08:33

TYP: Rename ModelT to ModelType

4863938

TYP: Use typing.overload and typing.get_overloads in py3.11+

f986316

Co-Authored-By: John Siirola <356359+jsiirola@users.noreply.github.com>

refactor: solver_factory: Promote the state to module-level variable

b5a45cb

style: reformat with black

f2db1c7

blnicho added the AT: PRE-TEST INSPECTED label Dec 12, 2025

jsiirola reviewed Dec 12, 2025

View reviewed changes

pyomo-autotest removed the AT: PRE-TEST INSPECTED label Dec 12, 2025

Revert "refactor: solver_factory: Promote the state to module-level…

fa1ade7

… variable" This reverts commit b5a45cb.

refactor: solver_factory: Promote the state to module-level variable

b42557a

jsiirola reviewed Dec 12, 2025

View reviewed changes

jsiirola approved these changes Dec 12, 2025

View reviewed changes

github-project-automation bot moved this from Todo to Reviewer Approved in Pyomo 6.10 Dec 12, 2025

jsiirola added the AT: PRE-TEST INSPECTED label Dec 12, 2025

fix: potentially bug if deleting first call

b4fcea0

Co-Authored-By: John Siirola <356359+jsiirola@users.noreply.github.com>

jsiirola added AT: PRE-TEST INSPECTED and removed AT: PRE-TEST INSPECTED labels Dec 12, 2025

pyomo-autotest removed the AT: PRE-TEST INSPECTED label Dec 12, 2025



		# NOTE: Python 3.11+ use `typing.Self`
		ModelT = TypeVar("ModelT", bound="Model")

TYP: Enhance static typing #3771

Are you sure you want to change the base?

TYP: Enhance static typing #3771

Conversation

n-takumasa commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes # NA

Summary/Motivation:

Changes proposed in this PR:

Legal Acknowledgement

Uh oh!

jsiirola left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jsiirola commented Dec 11, 2025

Uh oh!

n-takumasa commented Dec 12, 2025

Uh oh!

jsiirola left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsiirola left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

n-takumasa commented Oct 24, 2025 •

edited

Loading

codecov bot commented Nov 17, 2025 •

edited

Loading