Skip to content

Performance of typing._ProtocolMeta._get_protocol_attrs and isinstance #74690

Closed
#103348
Closed
Performance of typing._ProtocolMeta._get_protocol_attrs and isinstance#74690
#103348
Assignees
AlexWaygood
Labels
3.12only security fixesperformancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directorytopic-typing

Description

@orenbenkiki
orenbenkiki
mannequin
opened on May 29, 2017
BPO 30505
Nosy @ilevkivskyi, @orenbenkiki

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-05-29.09:16:49.884>
labels = ['library', 'performance']
title = 'Performance of typing._ProtocolMeta._get_protocol_attrs and isinstance'
updated_at = <Date 2017-06-02.17:44:59.311>
user = 'https://github.com/orenbenkiki'

bugs.python.org fields:

activity = <Date 2017-06-02.17:44:59.311>
actor = 'levkivskyi'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-05-29.09:16:49.884>
creator = 'orenbenkiki'
dependencies = []
files = []
hgrepos = []
issue_num = 30505
keywords = []
message_count = 2.0
messages = ['294686', '295044']
nosy_count = 2.0
nosy_names = ['levkivskyi', 'orenbenkiki']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue30505'
versions = ['Python 3.6']

Linked PRs

Activity

  1. orenbenkiki commented on May 29, 2017

    @orenbenkiki
    orenbenkikimannequin
    on May 29, 2017
    MannequinAuthor

    In 3.6.0, invocation of isinstance calls typing._ProtocolMeta._get_protocol_attrs.
    This creates a set of all attributes in all base classes, loops on these attributes to check they exist, and discards the set. It is very slow.

    My program uses isinstance to allow for flexibility in parameter types in certain key functions. I realize that using isinstance is frowned upon, but it seems to make sense in my case.

    As a result, >95% of its run-time is inside typing._ProtocolMeta._get_protocol_attrs (!).

    I have created a simple wrapper around isinstance which caches its result with a Dict[Tuple[type, type], bool]. This solved the performance problem, but introduced a different problem - type checking.

    I use mypy and type annotations, and my code cleanly type-checks (with the occasional # type: ignore). If I switch to using my own isinstance function, then mypy's type inference no longer treats it as special, so it starts complaining about all uses of values protected by if isinstance(value, SomeType): ...

    I propose that either the private typing._ProtocolMeta.__subclasscheck__ (which invokes _get_protocol_attrs), or the public isinstance, would be modified to cache their results.

    I can create a PR for either approach, if this is acceptable.

  2. orenbenkiki
    added
    stdlibStandard Library Python modules in the Lib/ directory
    performancePerformance or resource usage
    on May 29, 2017

  3. ilevkivskyi commented on Jun 2, 2017

    @ilevkivskyi
    ilevkivskyi
    on Jun 2, 2017
    Member

    Thanks for reporting!

    The runtime implementation of protocol classes will be thoroughly reworked as a part of PEP-544, see also python/typing#417 for a proof of concept runtime implementation.

    Also, there is another ongoing discussion python/typing#432 about a global refactoring of typing module that will significantly improve performance.

    Therefore, I would wait with any large PRs until these two stories are settled. If you still want to propose a small PR, you can do this at the upstream typing repo https://github.com/python/typing

  4. ezio-melotti
    transferred this issue fromon Apr 10, 2022
  5. AlexWaygood
    added
    topic-typing
    on Apr 13, 2022

  6. iritkatriel commented on Sep 7, 2022

    @iritkatriel
    iritkatriel
    on Sep 7, 2022
    Member

    @orenbenkiki @ilevkivskyi What is the status of this issue, five years on?

  7. ilevkivskyi commented on Sep 7, 2022

    @ilevkivskyi
    ilevkivskyi
    on Sep 7, 2022
    Member

    TBH I am not sure this issue is still relevant (at least I didn't hear any recent complaints about Protocol performance). @orenbenkiki is performance still bad for your application?

  8. AlexWaygood commented on Sep 7, 2022

    @AlexWaygood
    AlexWaygood
    on Sep 7, 2022
    Member

    I've seen recent complaints about the performance of Protocol from users of the beartype third-party library, and by @posita here. (I'm not sure I've ever seen a proper write-up of the performance problems in a CPython issue, however.)

  9. ilevkivskyi commented on Sep 7, 2022

    @ilevkivskyi
    ilevkivskyi
    on Sep 7, 2022
    Member

    Hm, actually the short-circuiting proposed in that comment may be a good idea (unless I forgot something) for a simple perf optimization (btw FWIW this is what mypy does internally, it always checks nominal subtypig first).

  10. AlexWaygood commented on Sep 7, 2022

    @AlexWaygood
    AlexWaygood
    on Sep 7, 2022
    Member

    @posita appears to have implemented optimised versions of ProtocolMeta here and here.

    (I haven't studied the code for either link, but may be worth looking at!)

  11. iritkatriel
    added
    3.12only security fixes
    on Sep 7, 2022

  12. posita commented on Sep 7, 2022

    @posita
    posita
    on Sep 7, 2022
    Contributor

    @posita appears to have implemented optimised versions of ProtocolMeta here and here.

    (I haven't studied the code for either link, but may be worth looking at!)

    For color, the current implementation lives in Beartype and is divided among two classes:

    The meat is in _CachingProtocolMeta whereas the Protocol is basically a hack to work around some standard library implementation details and could likely be eliminated. At a high level, _CachingProtocolMeta overrides __instancecheck__ to perform an examination of whether an object "satisfies" a Protocol definition similar to typing._ProtocolMeta.__instancecheck__ and then caches the result by that object's type. Note the asymmetry: the examination is performed on the object, but it is assumed that all objects of the first-examined object's type are consistent. This makes the implementation ill-suited for runtime-composed "types" or monkey-patched objects.

    Happy to discuss further, if desired.

  13. ilevkivskyi commented on Sep 7, 2022

    @ilevkivskyi
    ilevkivskyi
    on Sep 7, 2022
    Member

    Caching may need a more careful consideration (especially w.r.t. monkey-patching classes). I was specifically referring to this idea

    class _ProtocolMeta(ABCMeta):
    # …
    def __instancecheck__(cls, instance):
        if super().__instancecheck__(instance):
            # Short circuit for direct inheritors
            return True
        else:
            # … existing implementation that checks method names, etc. …
            return False

    @posita Do you have any good code base where you can benchmark this w.r.t. current implementation?

  14. posita commented on Oct 9, 2022

    @posita
    posita
    on Oct 9, 2022
    Contributor

    I can probably do something crude with numerary's and dyce's respective batteries of unit tests. Those are what motivated beartype.typing._typingpep544._CachingProtocolMeta in the first place. Give me a few days and I can probably throw something together.

  15. 43 remaining items

  16. carljm
    added a commit that references this issue on May 18, 2023

    Merge branch 'main' into nestedcomps

    54095f0
  17. JelleZijlstra
    added a commit that references this issue on May 21, 2023

    gh-74690: Make a typing test more resilient (#104691)

    b870b1f
  18. kraj
    added a commit that references this issue on Jun 1, 2023

    python3-typing-extensions: upgrade 4.5.0 -> 4.6.2

    d66bbb5
  19. rpurdie
    added a commit that references this issue on Jun 2, 2023

    python3-typing-extensions: upgrade 4.5.0 -> 4.6.2

    674b2c8
  20. halstead
    added a commit that references this issue on Jun 2, 2023

    python3-typing-extensions: upgrade 4.5.0 -> 4.6.2

    a37154b
  21. renovate
    added a commit that references this issue on Jun 3, 2023

    Update dependency typing-extensions to v4.6.3 (#224)

    b61076e
  22. AlexWaygood
    added 4 commits that reference this issue on Dec 4, 2023

    pythongh-74690: Optimise `isinstance()` and `issubclass()` calls agai…

    ec4051a

    gh-74690: Optimise `isinstance()` and `issubclass()` calls against ru…

    c718ab9

    pythongh-74690: Avoid a costly type check where possible in `_Protoco…

    35ba873

    gh-74690: Avoid a costly type check where possible in `_ProtocolMeta.…

    2ed20d3
  23. aisk
    added 2 commits that reference this issue on Feb 11, 2024

    pythongh-74690: Optimise `isinstance()` and `issubclass()` calls agai…

    78c8faf

    pythongh-74690: Avoid a costly type check where possible in `_Protoco…

    4fceb7f

  24. namka279 commented on Jul 21, 2024

    @namka279
    namka279
    on Jul 21, 2024 · Hidden as spam
  25. Glyphack
    added 2 commits that reference this issue on Sep 2, 2024

    pythongh-74690: Optimise `isinstance()` and `issubclass()` calls agai…

    cb075b6

    pythongh-74690: Avoid a costly type check where possible in `_Protoco…

    ed1e65f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixesperformancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directorytopic-typing

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions