MyPy forgets callable type guard in lambda in return statement of function with hinted return type

[lukas-lang]

Bug Report

There seems to be some strange interaction between hinted return types, lambdas and the callable type guard.

To Reproduce
Here is a minimal example, together with some related examples that are not broken, showing that the return type, lambda and callable are all required for this to break:

(Playground: https://mypy-play.net/?mypy=latest&python=3.11&gist=26f7a80bf3eff44ebf560ce5db694107)

from typing import Callable

def foo(x: int | Callable) -> int | Callable:
    if callable(x):
        return lambda: x()
    else:
        return x


def foo_fixed(x: int | Callable) -> int | Callable:
    if callable(x):
        ret = lambda: x()
        return ret
    else:
        return x


def foo_fixed2(x: int | Callable):
    if callable(x):
        return lambda: x()
    else:
        return x


def foo_fixed3(x: int | float) -> int | Callable[[], float]:
    if isinstance(x, float):
        return lambda: x
    else:
        return x


def foo_fixed4(x: int | Callable) -> float | Callable:
    if callable(x):
        return x
    else:
        return x + 1.0

Expected Behavior
Mypy should remember that x is a Callable within the lambda in the return statement.

Actual Behavior

MyPy forgets the type guard, and thinks that x is int | Callable in line 5:

main.py:5: error: "int" not callable  [operator]
Found 1 error in 1 file (checked 1 source file)

Your Environment

• Mypy version used: 1.5.0
• Mypy command-line flags: -
• Mypy configuration options from mypy.ini (and other config files): -
• Python version used: 3.11

[hauntsaninja]

Thanks for the issue!

The underlying reason is that closures in Python are late binding, see https://mypy.readthedocs.io/en/stable/common_issues.html#narrowing-and-inner-functions.

mypy recently improved heuristics here (see #2608), but I guess not quite well enough to avoid the false positive in your case.

[hauntsaninja]

Fixed by #16407