Break cyclic import between mypy.types and mypy.expandtype#21264
Open
rennf93 wants to merge 1 commit intopython:masterfrom
Open
Break cyclic import between mypy.types and mypy.expandtype#21264rennf93 wants to merge 1 commit intopython:masterfrom
rennf93 wants to merge 1 commit intopython:masterfrom
Conversation
`InstantiateAliasVisitor` was defined at the bottom of `mypy.types` and relied on a module-level back-import `from mypy.expandtype import ExpandTypeVisitor`. Current comment acknowledges this as "unfortunate". The back-import is load-order-dependent. When an external consumer (e.g. a mypy plugin such as `pydantic.mypy`) imports `mypy.expandtype` before `mypy.types` has finished loading, execution of `mypy/expandtype.py` line 8 re-enters `mypy/types.py`, which at module level then does `from mypy.expandtype import ExpandTypeVisitor` — but `ExpandTypeVisitor` has not been bound on `mypy.expandtype` yet. Under pure-Python mypy this raises ImportError: cannot import name 'ExpandTypeVisitor' from partially initialized module 'mypy.expandtype' Under compiled mypy (mypyc builds) the `AttributeError` is swallowed silently during plugin registration, so the plugin never loads and downstream decorators (`@field_validator`, `@model_validator`, etc.) are reported as `untyped-decorator` with no diagnostic hint. Fix: move `InstantiateAliasVisitor` to `mypy.expandtype` beside its base class, and replace the module-level back-import in `mypy.types` with a function-scoped lazy import at the single usage site (`TypeAliasType._partial_expansion`). No behavioural change — class body is identical and the one call site is semantically preserved. Verified: full `mypy/test/testcheck.py` suite (7833 tests) passes unchanged; `python -c "import pydantic.mypy"` succeeds on both compiled and pure-Python builds; plugin registers correctly when mypy runs.
Contributor
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
InstantiateAliasVisitorlives at the bottom ofmypy/types.pyand is defined via a module-level back-importfrom mypy.expandtype import ExpandTypeVisitor. The current in-source comment already acknowledges this as "unfortunate":The back-import is load-order-dependent. When an external consumer — notably any mypy plugin that does
from mypy.expandtype import …at module scope — importsmypy.expandtypebeforemypy.typeshas finished initialising, execution ofmypy/expandtype.pyline 8 re-entersmypy/types.py, which then runs thefrom mypy.expandtype import ExpandTypeVisitorline at module scope — butExpandTypeVisitorhas not been bound on themypy.expandtypemodule object yet.Symptoms:
Pure-Python mypy raises explicitly:
Compiled mypy (mypyc build) silently swallows the
AttributeErrorduring plugin registration. The plugin never loads and the user gets no diagnostic — downstream decorators registered by that plugin (@field_validator,@model_validator, etc.) are reported asuntyped-decorator.pydantic.mypyis the most common reproducer in the wild:from mypy.expandtype import expand_type, expand_type_by_instanceat the top of the plugin reliably triggers this against compiled mypy ≥ 1.17 (and presumably earlier — I verified 1.17, 1.18, 1.19, 1.20 all exhibit the silent-swallow behaviour on mypyc builds).How I hit this
Found while hardening
guard-core, a security engine library whoseSecurityConfigis a pydantic v2BaseModelwith ~6@field_validator/@model_validatordecorated methods. Strict mypy config (disallow_untyped_decorators = true) with the officially recommendedplugins = ["pydantic.mypy"]was in place and had been green on a prior mypy version. After a routine dependency refresh picked up compiled mypy 1.20.1, CI started reporting:mypy -vshowed no plugin load log entries and no error about plugin registration. The@field_validatordecorator was being seen as untyped, which is the exact behaviour pydantic's mypy plugin is supposed to prevent.Minimal repro
Isolating the import path confirms the cycle is the root cause:
Fix
InstantiateAliasVisitortomypy/expandtype.py, right alongside its base classExpandTypeVisitor. It keeps the same class body and uses the existingmypy.type_visitorimport already present inexpandtype.py.mypy/types.pywith a function-scoped lazy import at the single usage site —TypeAliasType._partial_expansion(the only placeInstantiateAliasVisitoris constructed). This removes the cycle entirely.No behavioural change: class body is byte-identical, single call site is semantically preserved.
Verification
python -c "import pydantic.mypy"now succeeds on both compiled and pure-Python builds.pydantic.mypyplugin registers correctly when invoked viaplugins = ["pydantic.mypy"]; decorator typing for pydantic v2BaseModelvalidators is restored in downstream projects.mypy/test/testtypes.py: 118 passed, 2 skipped (unchanged from baseline).mypy/test/testcheck.py: 7833 passed, 33 skipped, 7 xfailed, 0 failed (unchanged from baseline).mypy_self_check.iniself-check: 34 errors identical to baseline (zero delta — the remaining 34 are pre-existing in test tooling, unrelated to this change).Test plan
python -m pytest mypy/test/testtypes.pypython -m pytest mypy/test/testcheck.pypython -m mypy --config-file mypy_self_check.ini -p mypy(diff vs baseline)python -c "import pydantic.mypy"(compiled and pure-Python builds)plugins = ["pydantic.mypy"]on a pydantic v2 BaseModel now correctly recognises@field_validatoras typed