Typing of return type of a generic function #8946

Activity

1. tarcisioe commented on Jun 5, 2020

   

   tarcisioe

   on Jun 5, 2020

   Last edited by tarcisioe

   More actions

   I have made a very similar observation on #8941, but since Optional[str] and Type[str] are not reified types, they shouldn't be used like that. I find that somewhat troubling, because:

   1 - It impedes the use you are showing, of specializing the return type of a function, unless T is a runtime-valid type (List[...] works, for example).
   2 - Makes code that use introspection much harder to type, leaving it to be very sparsely type-checked (if at all), having to rely on other clunky techniques.

   In my case, I ended up (for Optional), using an overload with optional as a bool argument, and overloading on Literal[True] and Literal[False].

   And by the way, 0.770 deduced the type as Any, not as Optional[str], in your first example. I found that out by using reveal_type after 0.780 started showing the error.

   In the end, I still need to use Optional in runtime to be able to validate input into dataclasses, (using typing_inspect to help), but I can't write tests with code that calls functions literally with Optional[Something] unless I #type: ignore the line.
2. JukkaL commented on Jun 8, 2020

   

   JukkaL

   on Jun 8, 2020

   Collaborator

   More actions

   As explained by @tarcisioe, the new behavior better aligns with runtime Python behavior.
3. JukkaL

   closed this as completedon Jun 8, 2020
4. ltworf commented on Jun 14, 2020

   

   ltworf

   on Jun 14, 2020

   More actions

   So just out of curiosity, forgetting the entire Optional thing, say that I have this function:

   def create(t: Type[T]) -> T:
       return t()
   
   a = create(int)
   b = create(str)

   What should the types be to let mypy know that a is an int and b is a str? I had always assumed this to work but as @tarcisioe it does not.
5. andersk commented on Jun 14, 2020

   

   andersk

   on Jun 14, 2020

   Contributor

   More actions

   @ltworf def create(t: Callable[[], T]) -> T.
6. ltworf commented on Jun 14, 2020

   

   ltworf

   on Jun 14, 2020

   Last edited by ltworf

   More actions

   It seems incorrect, or well it works for those but not if I start passing like List[int] which works with Type[T].

   from typing import *
   T = TypeVar('T')
   def create(t: Type[T]) -> T:
       ...
   
   reveal_type(create(int))
   reveal_type(create(str))
   reveal_type(create(List[int]))
   reveal_type(create(Dict[int, int]))
   reveal_type(create(Tuple[int]))
   reveal_type(create(Union[int,str]))
   reveal_type(create(Optional[int]))

   Result:

   testo2.py:7: note: Revealed type is 'builtins.int*'
   testo2.py:8: note: Revealed type is 'builtins.str*'
   testo2.py:9: note: Revealed type is 'builtins.list*[builtins.int*]'
   testo2.py:10: note: Revealed type is 'builtins.dict*[builtins.int*, builtins.int*]'
   testo2.py:11: note: Revealed type is '<nothing>'
   testo2.py:11: error: Argument 1 to "create" has incompatible type "object"; expected "Type[<nothing>]"
   testo2.py:12: note: Revealed type is '<nothing>'
   testo2.py:12: error: Argument 1 to "create" has incompatible type "object"; expected "Type[<nothing>]"
   testo2.py:13: note: Revealed type is '<nothing>'
   testo2.py:13: error: Argument 1 to "create" has incompatible type "object"; expected "Type[<nothing>]"
   Found 3 errors in 1 file (checked 1 source file)
   

   the Type[T] thing works but only for some types and not others. It is very strange really.

   I could understand it not working for union types, but why does it also fail for Tuple, but work for List and Dict???

   I was using this for my typedload module with a load(data, type: Type[T]) -> T signature that turns out works for some types but not for some others, and the new mypy version revealed this.

   Should I open a different bug to track this?

   👍React with 👍1Reacted by Tarcisio
7. tarcisioe commented on Jun 14, 2020

   

   tarcisioe

   on Jun 14, 2020

   Last edited by tarcisioe

   More actions

   @ltworf I think this won't be treated as a bug. Since #8941 and this issue were closed for the same reason, this seems like the intended behaviour. But it is a missing feature.

   Basically what we are looking for is something mirroring what could be written in many other languages with generics as something along the lines of (pardon the C++)

   template <typename T>
   T load(std::string data) {}  // std::string for lack of a better type here

   Unfortunately, there is no real syntax for that. The closest you can get is by using a class just to get the generic:

   T = TypeVar('T')
   
   class Converter(Generic[T]):
       @staticmethod
       def convert(data: Any) -> T:
           ...

   But there are two issues here: this is pretty verbose, and there is no (easy, at least) runtime access to whatever T was, as far as I am aware, so I don't see that as a very good solution. If there is a way to get the real T at runtime I would find it acceptable, though I'd rather look at something terser.
8. ltworf

   added a commit that references this issue on Aug 24, 2020

   Usual mypy workaround

   ...

   bd1c83a