1

Consider the following minimal example in which we have a function foo that can be either called with one int and one string or with two strings:

from typing import overload

@overload
def foo(x: int, y: str) -> str: ...

@overload
def foo(x: str, y: int) -> str: ...

@overload
def foo(x: str, y: str) -> int: ...

def foo(x, y):
    if isinstance(x, int):
        return str(x - 2) + y
    if isinstance(y, int):
        return str(y - 2) + x
    return int(x + y)

While the implementation of foo should be fine, it is not checked by mypy. So if we should have any type error in the function body, it will not be caught.

So here is my question: Why doesn't mypy simply check the consistency of every signature with the implementation? And how could we make mypy check every signature? Clearly, we can't annotate the implementation like this:

def foo(x: str | int, y: str | int) -> str | int:      # mypy complains rightfully
    pass

Neither do I see how we could a TypeVar to resolve the issue. So what should we do?

Some discussion of this can be found in this GitHub issue and in this question. Neither provides a solution for the problem I have described or a satisfying justification for why type checking is "too challenging" for overloaded functions.

Improve this question
7
  • 1
    Well, it's quite challenging. Assume you have 10 overloads, differing in keyword/positional parameters, maybe accepting **kwargs somewhere, and having partially overlapping types (but not fully, so that every overload is indeed well-defined). Then you need to analyze the function body, and on every call/assignment/return check which of the overloads are in use, and whether the contract is met. If you think it's easy - you may try, mypy always appreciates contributions. I tried once and gave up after ~800 lines of code and even haven't started adding TypeVar support there. Commented Jun 14, 2023 at 23:47
  • 1
    To just get body typechecking, you should annotate the function with "union" of the overloads (it's not a real union, of course - overload is a union from type theory perspective - but more like a distribution of union operation over args). You'll get false positive for combinations that are impossible due to your overloads, but it's better than ignoring the body completely. Commented Jun 14, 2023 at 23:49
  • Also I suppose that questions "Why does not the library X do Y" are off-topic, because they either lead to opinion-based answers or can be answered only by library maintainers and are unhelpful to remaining site audience. You have even found an issue with GVanRossum explanation (he knows better), and a question suggesting alternatives. What kind of answer do you expect? Commented Jun 14, 2023 at 23:53
  • Thanks @SUTerliakov I appreciate your answers a lot :) I am still a little puzzled how I could properly type-check my minimal example though. As I pointed out, annotating the body with the "union of the overloads" will give mypy errors, due to the false positives (that you also noted in your comment). So my main concern is how should approach this type of problem (don't type check at all, use # type ignore, or is there maybe something smarter we can do?) Commented Jun 15, 2023 at 9:33
  • 1
    Sorry, probably i misunderstood your wording. I'd annotate the implementation and then verify call contract with isinstance (asserts can be optimized away with python interpreter flag, if performance is a real concern here), something like this. Commented Jun 15, 2023 at 10:25

1 Answer 1

Reset to default
0

I see several issues here:

  1. your type overloads do not strictly match the use of isinstance. E.g., in the first case if isinstance(x, int), the corresponding overload states that y is a string, but that's not enforced/documented anywhere.

  2. in general, if you have simple combinations of types, TypeVar works quite well, but in this case you have a "mapping" from input types to output types, which I don't think mypy can handle at all.

An example of a simple case would be:

from typing import TypeVar

T = TypeVar("T", int, str)

def foo(x: T, y: T) -> T:
    if isinstance(x, int):
        # you might want to assert isinstance(y, int)
        return x * y
    elif isinstance(x, str):
        return x + y
    else:
        raise TypeError(...)

Long story short, I think in your case the solution is not using overrides, and instead using specialized functions/methods for each combination of types, and a wrapping function that figures out which specialized function it should call:

def foo(x: int | str, y: int | str) -> int | str:
    # you are obviously losing the "link" between the combination of input
    # types and the output type. If you care about preserving that, call one
    # the appropriate specialized function directly.
    if isinstance(x, int):
        assert isinstance(y, str)
        return foo_a(x, y)
    if isinstance(y, int):
        return foo_b(x, y)
    # mypy should figure out they are both strings
    return foo_c(x, y)


def foo_a(x: int, y: str) -> str:
    return str(x - 2) + y


def foo_b(x: str, y: int) -> str:
    return str(y - 2) + x


def foo_c(x: str, y: str) -> int:
    return int(x + y)
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.