Fix overload ambiguity returning Any when one overload returns Never

[LeSingh1]

Fixes #21027.

When checking an overloaded call where the argument has type Any, mypy collects all matching overloads and checks whether the Any causes ambiguity. If it does and the return types differ, mypy falls back to Any. This is the right behavior in general, but it is wrong when one of the matching overloads returns Never.

Never is a subtype of every type. An overload returning Never is always the most specific result possible. If exactly one overload in the ambiguous set returns Never, there is no actual ambiguity to resolve: the Never overload is the right answer regardless of which overload would have been selected at runtime.

The bug was visible with iter() on generic classes that have both __iter__ (returning Never) and __getitem__. The iter() builtin has two relevant overloads: one matching SupportsIter (uses __iter__) and one matching _GetItemIterable (uses __getitem__). When the class had an Any type argument, both overloads matched, ambiguity was detected, and iter(x) returned Any instead of Never.

Calling x.__iter__() directly returned Never correctly; only iter(x) was broken.

Before this fix:

from typing import Any, Generic, Never, TypeVar, reveal_type

T1 = TypeVar("T1")
T2 = TypeVar("T2", bound=object)

class C(Generic[T1, T2]):
    def __getitem__(self, key: int) -> T2: ...
    def __iter__(self) -> Never: ...

def test(x: C[Any, str]) -> None:
    reveal_type(x.__iter__())  # Never
    reveal_type(iter(x))       # Any  <-- wrong

After this fix:

    reveal_type(iter(x))       # Never  <-- correct

The fix is in infer_overload_return_type in mypy/checkexpr.py. After detecting Any-caused ambiguity, before falling back to Any, it checks whether exactly one of the matching overloads returns Never. If so, it returns that overload's result instead.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

optuna (https://github.com/optuna/optuna)
+ optuna/importance/_fanova/_tree.py:187: error: Need type annotation for "sizes" (hint: "sizes: list[<type>] = ...")  [var-annotated]
+ optuna/importance/_fanova/_tree.py:199: error: Need type annotation for "size"  [var-annotated]
+ optuna/importance/_fanova/_tree.py:306: error: Incompatible return value type (got "signedinteger[_32Bit | _64Bit]", expected "float")  [return-value]
+ optuna/_hypervolume/wfg.py:12: error: Need type annotation for "edge_length_y"  [var-annotated]
+ optuna/_hypervolume/wfg.py:35: error: Need type annotation for "x_delta"  [var-annotated]
+ optuna/_hypervolume/wfg.py:58: error: Need type annotation for "inclusive_hvs"  [var-annotated]
+ optuna/_hypervolume/hssp.py:115: error: Need type annotation for "diff_of_loss_vals_and_ref_point"  [var-annotated]
+ optuna/samplers/_tpe/sampler.py:669: error: Need type annotation for "acq_func_vals"  [var-annotated]
+ optuna/visualization/_optimization_history.py:281: error: Need type annotation for "lower"  [var-annotated]
+ optuna/visualization/matplotlib/_optimization_history.py:135: error: Need type annotation for "lower"  [var-annotated]
+ tests/gp_tests/test_batched_lbfgsb.py:36: error: Incompatible return value type (got "tuple[Any, float]", expected "tuple[ndarray[tuple[Any, ...], dtype[Any]], ndarray[tuple[Any, ...], dtype[Any]]]")  [return-value]

spark (https://github.com/apache/spark)
+ python/pyspark/worker.py:2826: error: Need type annotation for "value_df"  [var-annotated]

colour (https://github.com/colour-science/colour)
+ colour/geometry/intersection.py:214: error: Need type annotation for "x_4_x_3"  [var-annotated]
+ colour/geometry/intersection.py:215: error: Need type annotation for "y_1_y_3"  [var-annotated]
+ colour/geometry/intersection.py:216: error: Need type annotation for "y_4_y_3"  [var-annotated]
+ colour/geometry/intersection.py:217: error: Need type annotation for "x_1_x_3"  [var-annotated]
+ colour/geometry/intersection.py:218: error: Need type annotation for "x_2_x_1"  [var-annotated]
+ colour/geometry/intersection.py:219: error: Need type annotation for "y_2_y_1"  [var-annotated]
+ colour/geometry/intersection.py:221: error: Need type annotation for "numerator_a"  [var-annotated]
+ colour/geometry/intersection.py:222: error: Need type annotation for "numerator_b"  [var-annotated]
+ colour/geometry/intersection.py:223: error: Need type annotation for "denominator"  [var-annotated]
+ colour/geometry/intersection.py:232: error: Need type annotation for "parallel"  [var-annotated]
+ colour/difference/delta_e.py:1135: error: Need type annotation for "dLab"  [var-annotated]
+ colour/characterisation/correction.py:1454: error: Need type annotation for "D"  [var-annotated]
+ colour/quality/tm3018.py:206: error: Need type annotation for "a_deltas"  [var-annotated]
+ colour/quality/tm3018.py:207: error: Need type annotation for "b_deltas"  [var-annotated]

altair (https://github.com/vega/altair)
+ tests/vegalite/v6/test_api.py:748: error: Need type annotation for "input_condition"  [var-annotated]

scipy (https://github.com/scipy/scipy)
+ scipy/stats/_multicomp.py:171: error: Need type annotation for "diff_means"  [var-annotated]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/series/test_truediv.py:180: error: Expression is of type "Never", not "Any"  [assert-type]
+ tests/series/test_sub.py:94: error: Expression is of type "Never", not "Any"  [assert-type]
+ tests/series/test_sub.py:97: error: Statement is unreachable  [unreachable]
+ tests/series/test_floordiv.py:148: error: Expression is of type "Never", not "Any"  [assert-type]
+ tests/series/test_floordiv.py:153: error: Statement is unreachable  [unreachable]
+ tests/series/test_floordiv.py:159: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/series/test_floordiv.py:165: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/series/test_floordiv.py:176: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/series/test_floordiv.py:177: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/indexes/test_sub.py:66: error: Expression is of type "Never", not "Any"  [assert-type]
+ tests/indexes/test_sub.py:69: error: Statement is unreachable  [unreachable]
+ tests/indexes/test_floordiv.py:115: error: Expression is of type "Never", not "Any"  [assert-type]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/indexes/base.py:4179: error: Need type annotation for "diff"  [var-annotated]

xarray (https://github.com/pydata/xarray)
+ xarray/tests/test_cftime_offsets.py: note: In function "test_cftime_range":
+ xarray/tests/test_cftime_offsets.py:1272: error: Need type annotation for "deltas"  [var-annotated]
+ xarray/tests/test_cftime_offsets.py:1273: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "Never")  [assignment]
+ xarray/tests/test_cftime_offsets.py:1273: error: Need type annotation for "delta"  [var-annotated]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/lowering.py:4291: error: Need type annotation for "sizes"  [var-annotated]