AnyOf - Union for return types

Sometimes a function or method can return one of several types, depending on the passed in arguments or external factors. Best example is probably open(), but there are other examples in the standard library, like shutil.copy()[1]. In many of those cases, the caller knows what return type to expect.

Currently there are several options, but none of them is really satisfactory:

Use @overload. This is the best solution if it can be used. But that is often not the case, like in the examples above.
Use a mypy plugin. This solution does not scale to outside the standard library (and arguably not even inside it) and is mypy-specific.
Use Union as the return type. This is usually not recommended, since it means that the caller needs to use isinstance() to use the return type.
Use Any as the return type. This is currently best practice in those cases, but of course provides no type safety at all.

Therefore, I propose to add another type, for example AnyOf[...] that acts like Union, but can be used everywhere any of its type arguments could be used.

from datetime import date
x: AnyOf[str, date] = ...
s: str = x  # ok
dt: date = x  # ok
i: int = x  # type error
u: Union[str, bytes] = x  # ok
x = u  # type error (although the type checker could do something smart here and infer that u can only be str here)

I also think that the documentation should make it clear that using AnyOf is a code smell.

[1] Currently the type behaviour in shutil is broken in my opinion, but that does not change the fact that currently it is as it is.

Another use case - at least util we get a more flexible Callable syntax - is for optional arguments in callback functions, like WSGI's start_response():

StartResponse = AnyOf[
    Callable[[str, List[(str, str)]], None],
    Callable[[str, List[(str, str)], ExcInfo], None]
]

def create_start_response() -> StartResponse:
    ...

create_start_response()("200 OK", [])

Using Union this causes a type error. (Too few arguments.)

This would be as unsafe as Any though right? E.g. def foo() -> Union[int, str] -- we have no idea whether foo() + 1 is safe or not. Sure, it can tell you that foo().append(1) is definitely wrong, but that's pretty minor.

Similarly we won't know if create_start_response()("200 OK", [], sys.exc_info()) will be accepted or not. If you meant to describe a situation where it returns a callback that can be called with or without the extra argument, there's already a solution: https://mypy.readthedocs.io/en/latest/additional_features.html?highlight=mypy_extensions#extended-callable-types.

Good to know that there is a proper solution for the callback case!

Personally, I think the improvement in type safety over just returning Any would be worth it. It surely can't catch all problematic cases, but some is better than none. And the issue seems to crop up from time to time, for example just after I opened this in python/typeshed#2271. That the problem of returning Unions is also explicitly mentioned in the docs is also noteworthy, I think.

In my experience I never had a situation where I needed unsafe unions. Anyway, I could imagine some people might want it. However, the problem is that the benefits of unsafe unions are incomparable with the amount of work to implement them. Adding a new kind of types to mypy e.g. could take months of intense work. This one will be as hard as introducing intersection types, and while the later are more powerful (they cover most of use cases of unsafe unions) we still hesitate to start working on it.

I'm with Ivan, and this is something we've considered earlier -- see the discussion at python/mypy#1693, for example. The relatively minor benefits don't really seem worth the extra implementation work and complexity.

This would only be a potentially good idea for legacy APIs that can't be typed precisely right now, and the most popular of those can be special cased by tools (e.g. through mypy plugins). Mypy already special cases open and a few other stdlib functions. Alternatively, we might be able to use some other type system improvements, such as literal types, once they become available. For new code and new APIs the recommended way is to avoid signatures that would require the use of AnyOf anyway.

Ad-hoc extensions have the benefit of being easy to implement. They are also modular, don't complicate the rest of the type system, and they potentially allow inferring precise return types.

There is also often a simple workaround -- write a wrapper function around the library function with an Any return that has a precise return type, by restricting the arguments so that the return type can be known statically. Example:

def open_text(path: str, mode: str) -> TextIO:
    assert 'b' not in mode
    return open(path, mode)

def open_binary(path: str, mode: str) -> BinaryIO:
    assert 'b' not in mode
    return open(path, mode + 'b')

Some use cases (such as open) can now be supported pretty well by using overloads and literal types (PEP 586).

This continues to crop up with typeshed pretty regularly. For a lot of return types, typeshed either has to either make a pretty opinionated decision or completely forgo type safety with Any.

One use case I find pretty compelling is for autocomplete in IDEs, eg: python/typeshed#4511
I seem to recall PyCharm used unsafe unions and I'd imagine this is a big reason why.

From a typeshed perspective, it would be nice to support these use cases. From a mypy perspective, I agree it's maybe not worth the effort, so maybe type checkers could interpret a potential AnyOf as a plain Any.

Maybe you could bring this up on typing-sig? A proto-pep might get support there.

Or maybe you can spell this using -> Annotated[Any, T1, T2] where T1, T2 are the possible return types? Then type checkers will treat it as Any but other tools could in theory interpret this as AnyOf[T1, T2]. Or is that too hacky?

I brought this up on typing-sig.

Semantically, would AnyOf be equivalent to Intersection as proposed in #213? My intuition is yes: an operation on an AnyOf type should be valid if it is valid on at least one of the component types.

Guido's thoughts on the subject: https://mail.python.org/archives/list/typing-sig@python.org/message/TTPVTIKZ6BFVWZBUYR2FN2SPGB63Z7PH/
~~edited out misleading tldr~~

There's probably also some slightly different behaviour when intersecting slightly incompatible types. E.g., for an intersection maybe you'd want to treat intersection order like an MRO, but for AnyOf you'd probably want "is compatible with any of the intersection"

I see, thanks for reminding me of that email! I suppose this matters when you're implementing a function with an AnyOf return type. In typeshed we could just write:

def some_func() -> Intersection[bytes, str]: ...

And it would work as expected.

But when implementing it, you'd write:

def some_func() -> Intersection[bytes, str]:
    if something:
        return bytes()
    else:
        return str()

And a type checker would flag the return type as incompatible. So in Guido's terminology, AnyOf would have to behave like Union in a "receiving" position and like Intersection in a "giving" position.

Maybe I'm misunderstanding how intersections are supposed to work, but to me an intersection type is a type that fulfills the protocol of multiple other types. At least that's how e.g. typescript and the Ivan's initial comment in #213 describe it. Intersection[bytes, str] wouldn't make much sense to me, because it would mean that the returned type is both a str and a bytes. An intersection lets you "compose" multiple types into one, which is why I like the Foo & Bar syntax for it (also like typescript and in comparison to | for union).

And that means that AnyOf has not much relation to intersections. Like Union, it's more meant to be an "either/or" situation. For example, the following would work with AnyOf, but not with Union (which why AnyOf is unsafe, but still much safer than Any):

def foo(b: bytes): ...

x: AnyOf[str, bytes]
y: str | bytes
foo(x)  # ok
foo(y)  # error

And sorry for the spam, but one last thought:

For the caller of a function, there is no difference, whether an argument is annotated with AnyOf of Union. In fact, I can't think of a reason why an argument should ever be annotated with it. It's mostly a tool for return types.

what if instead of introducing more Anys, we just address the original limitations of being unable to type certain functions?

@JelleZijlstra

what if instead of introducing more Anys, we just address the original limitations of being unable to type certain functions?

This will reduce the amount of "anyness" by limiting it to a subset of the possible types it can assume. So although @JelleZijlstra s intersection trick looks more "any-ish", it achieves the opposite.

But if you have a proposal that would avoid having to use gradual typing altogether, then I would certainly prefer that!

Kotlins type system supports this feature, named "flexible types", although they are specified with an upper and lower bound, not a set of types

Two use-cases for AnyOf, that as a nice side effect would simplify some internal logic of mypy:

In calls like f("x") the type of "x" can be str or Literal["x"]. Currently, mypy uses a special "last_known_value" mechanism to deal with this, but I think AnyOf[str, Literal["x"]] would achieve the same
tuple[T, ...] can in some contexts be treated like AnyOf[tuple[()], tuple[str], tuple[str, str], ...] rather than an infinite union, for examples:

def foo(x: str, y: str) -> None: ...
def test(arg: tuple[str, ...]) -> None:
    foo(*arg)

This call would be invalid if we treat tuple[str, ...] as a union, because the signature of foo is essentially equivalent to *args: *tuple[str, str], and Union[tuple[()], tuple[str], tuple[str, str], ...] is not a subtype of tuple[str, str]. But if we instead treat it as AnyOf[tuple[()], tuple[str], tuple[str, str], ...] the call is valid.

This will reduce the amount of "anyness"

It only reduces the amount of "anyness" if you assume that people would only use Any as the alternative when AnyOf is not available.
I think that's not a realistic assumption. People will more often choose unions when AnyOf is not available.

Any requires an import and union | doesn't.
If they would use AnyOf, they're probably already thinking of multiple specific types - leading them to think of unions and not Any.

So adding AnyOf would increase the amount the amount of "anyness", not decrease it.

People will more often choose unions when AnyOf is not available.

Unions are often unsuitable for return and other output types. See the many references to other issues in this thread.

@jorenham

I was personally not a fan of AnyOf initially, but now that I have seen what amount of trouble ambiguous overloads can cause (that return Any), I think it might be worth introducing AnyOf.

The problem currently is that both Mypy and Pyright don't follow the overload spec. Pyright doesn't work properly with ambiguous overloads at all and just picks the first winner, while Mypy doesn't return Any on ambiguous overloads, but some kind of merged fallback type that wouldn't for example return list[int] | list[str] in case of two overloads returning different lists, but return list[Any] instead.

Pyright currently doesn't get much love and Mypy is probably unlikely to change their behavior since that would cause a lot of trouble with some of the stricter typing options.

This leads us to the weird position where the newer type checkers - zuban, pyrefly and ty - are more correct, but also return much more Any than mypy/pyright, which is a problem for multiple reasons:

Type checkers typically have more strict options to detect if Any is returned or assigned somewhere. So if moving from Mypy/Pyright, suddenly lots of Any appear.
This is especially problematic for libraries like scipy-stubs that try to have overloads working for all type checkers, which is impossible, since they don't all follow the spec, but AnyOf would help in that regard. @jorenham (scipy-stubs maintainer) told me that they want Mypy/Pyright compatibility, since they are probably still the most used type checkers. Some of the workarounds involve trying to filter out Any matches with Never (e.g. tuple[Never] is matched only by tuple[Any]), which always leads to Any as an ambiguous fallback.

I feel like AnyOf could make overload fallbacks better, safer and more consistent for different type checkers.

@davidhalter

@davidhalter What do you think about the other use case I mentioned regarding literal values?

x = "x"
reveal_type(x)  # why not AnyOf[Literal["x"], str]?

Currently, type checkers can't seem to agree whether this x is a valid argument for a function that expects Literal["x"]:

from typing import Literal

def f(arg: str) -> None: ...
def g(arg: Literal["x"]) -> None: ...

f("x")  # OK
g("x")  # OK

x = "x"
f(x)  # OK
g(x)  # mypy/zuban error, pyright/pyrefly/ty pass

@randolf-scholz

@randolf-scholz In what way would the behavior of a type checker differ if it inferred AnyOf[Literal["x"], str] there, compared with simply inferring Literal["x"]?

@randolf-scholz

@randolf-scholz I think that Zuban/Mypy are wrong to not narrow x = "x" correctly. x can be inferred as str when accessed by an import, but it should be narrowed to an (implicit) literal in the local scope. I personally intend to change that for Zuban, but it's not that easy for a couple of reasons and as far as I know Mypy devs would also like to change that, since there are no real benefits of not narrowing simple assignments.

Mypy and Zuban both have the concept of an implicit Literal vs. explicit Literal, but this is not relevant for type checking, it is only relevant for type inference (e.g. Mypy infers list[int] for implicit Literals vs list[Literal[1]] for explicit literals if I remember correctly). AnyOf is mostly a different concept I would say.

@carljm

@randolf-scholz In what way would the behavior of a type checker differ if it inferred AnyOf[Literal["x"], str] there, compared with simply inferring Literal["x"]?

I think the only place I have seen this differ is when using assert_type(x, str) where x is an "implicit" literal. In that case x is essentially not defined from a type inference perspective and could become both types. Zuban used to work like that, but after problems with conformance tests I changed it to only assert the literal (since it is in fact a literal... but not really).

as far as I know Mypy devs would also like to change that

Yes, we are going to allow that (i.e. g(x)), hopefully in near future.

@carljm

@carljm most type checkers seem to use heuristics for when to convert literal types to non-literals. Potentially using AnyOf could reduce the amount of heuristics needed. Since I can't think of anything better right now, here's a somewhat contrived example:

from typing import Literal

def f(arg: str) -> None: ...
def g(arg: Literal["x"]) -> None: ...

x = "x"
t = (x,)   # tuple[Literal["x"]] or tuple[str]?
l = [x]    # list[Literal["x"]] or list[str]?
s = {x}    # set[Literal["x"]] or set[str]?

g(*t)  # pyrefly/ty/pyright: pass, zuban/mypy: error
g(*l)  # all error
g(*s)  # all error

If one wanted to go "gradual guarantee all the way", then all of these calls should succeed, because they do if one explicitly annotates x: Literal["x"] = "x".

If one wanted to go "gradual guarantee all the way", then all of these calls should succeed, because they do if one explicitly annotates x: Literal["x"] = "x".

Yes, that's true. I think ty has experimented further in the gradual guarantee direction than any other type checker, and our (or at least my) conclusion is that it's not really workable for the Python type system -- it requires too much annotation verbosity to achieve strict typing. (It could perhaps be workable for an "I just want an LSP, not strict typing" mode -- but in practice that mode is easier to achieve by silencing diagnostics than by implementing different inference rules.)

most type checkers seem to use heuristics for when to convert literal types to non-literals

I think (and this is supported by your example) this heuristic really only matters when inferring types for invariant containers. There is never any harm in inferring Literal["x"] over str, but there can be harm in inferring list[Literal["x"]] over list[str], due to invariance of list. So I think it would be clearer to focus the proposal on invariant-container inference. list[AnyOf[Literal["x"], str]] does seem like a potentially useful type. Although (c.f. the previous paragraph) I am not convinced that inferring this gradual type (and requiring everyone who wants strict typing to annotate always) is a better choice than just picking a non-gradual inference heuristic that seems to do well in the common cases, and then requiring explicit annotation for the unusual case (which would clearly be list[Literal["x"]] in this case.)

IMO ambiguous overloads (where AnyOf is a clear improvement over Any) is the best use case for AnyOf.

One defensive trick you can use for cases like l = ["x"] that could be inferred as either list[Literal["x"]] or list[str] is this:

def f[StrT: str](strs: list[StrT], /) -> None: ...

So here, StrT is a free type variable, which gives us a bit of wiggle room (limited to the scope of f).

BTW, this trick can also come in handy when the list item type can be e.g. either str or bytes, allowing you to write list[T] instead of list[str] | list[bytes] | list[str | bytes] :)

So as you can see, this free type variable trick is pretty similar to how you'd use AnyOf in input positions, especially if you use constraints instead of bounds. We'll still need AnyOf for output positions though, so we're only halfway there 😜.

@davidhalter

I think @davidhalter described the problems with ambiguous overloads well. Pyrefly has run into the same issues with producing lots of Any types on scipy-stubs overloads. I've made an alternate proposal here for a fallback that's less aggressive than Any: https://discuss.python.org/t/spec-change-ambiguous-arguments-in-overload-call-evaluation/106891. Compared to AnyOf, IMO this would be easier to adopt today because it doesn't require any new concepts or constructs (in fact, I've already implemented it in Pyrefly).

Navigation Menu

Repository navigation

AnyOf - Union for return types #566

14 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

AnyOf - Union for return types #566

Description

Activity

srittau commented on Jun 21, 2018

gvanrossum commented on Jun 21, 2018

srittau commented on Jun 21, 2018

ilevkivskyi commented on Jun 22, 2018

JukkaL commented on Jun 22, 2018

JukkaL commented on Jul 17, 2019

hauntsaninja commented on Sep 20, 2020

gvanrossum commented on Sep 20, 2020

srittau commented on Sep 21, 2020

JelleZijlstra commented on Aug 11, 2021

hauntsaninja commented on Aug 11, 2021

JelleZijlstra commented on Aug 11, 2021

srittau commented on Aug 11, 2021

srittau commented on Aug 11, 2021

srittau commented on Aug 11, 2021

14 remaining items

KotlinIsland commented on Jun 15, 2025

jorenham commented on Jun 15, 2025

KotlinIsland commented on Aug 1, 2025

randolf-scholz commented on Sep 12, 2025

beauxq commented on Nov 30, 2025

srittau commented on Nov 30, 2025

davidhalter commented on Apr 7, 2026

randolf-scholz commented on Apr 7, 2026

carljm commented on Apr 7, 2026

davidhalter commented on Apr 7, 2026

ilevkivskyi commented on Apr 7, 2026

randolf-scholz commented on Apr 7, 2026

carljm commented on Apr 7, 2026

jorenham commented on Apr 7, 2026

rchen152 commented on Apr 8, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions