Skip to content

incremental: Crash after deleting a function. #3052

Closed
Closed
incremental: Crash after deleting a function.#3052
Labels
crashtopic-incremental

Description

@timabbott
timabbott
opened on Mar 24, 2017

I ran into this crash with 0.501 today on the Zulip codebase when running mypy with this version of the code zulip/zulip@09f66b5 (which deleted the below function).

I've attached the cache state tarball in case it's helpful: mypy.tar.gz

Traceback (most recent call last):
  File "/srv/zulip-py3-venv/bin/mypy", line 6, in <module>
    main(__file__)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/main.py", line 42, in main
    res = type_check_only(sources, bin_dir, options)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/main.py", line 86, in type_check_only
    options=options)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 183, in build
    dispatch(sources, manager)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1531, in dispatch
    process_graph(graph, manager)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1761, in process_graph
    process_fresh_scc(graph, prev_scc)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1830, in process_fresh_scc
    graph[id].fix_cross_refs()
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1311, in fix_cross_refs
    fixup_module_pass_one(self.tree, self.manager.modules)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 21, in fixup_module_pass_one
    node_fixer.visit_symbol_table(tree.names)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 79, in visit_symbol_table
    stnode = lookup_qualified_stnode(self.modules, cross_ref)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 255, in lookup_qualified_stnode
    assert key in names, "Cannot find %s for %s" % (key, name)
AssertionError: Cannot find do_set_realm_description for zerver.lib.actions.do_set_realm_description

Activity

  1. refi64 commented on Mar 27, 2017

    @refi64
    refi64
    on Mar 27, 2017
    Contributor

    FWIW, I couldn't actually reproduce this with (inside the Zulip directory):

    rm -rf var/mypy_cache
git checkout 09f66b5c6dfc5c8bf5d52fb51259189ade0f336e^
tools/run-mypy
git checkout 09f66b5c6dfc5c8bf5d52fb51259189ade0f336e
tools/run-mypy

  2. gvanrossum commented on Mar 27, 2017

    @gvanrossum
    gvanrossum
    on Mar 27, 2017
    Member

    Yeah the missing info is all the mtimes of the files in Tim's working directory when it failed. :-(

  3. gvanrossum
    added
    crash
    topic-incremental
    on Aug 2, 2017

  4. gvanrossum commented on Aug 2, 2017

    @gvanrossum
    gvanrossum
    on Aug 2, 2017
    Member

    (FWIW if this occurs again on an open source package, as of mypy 0.521 the mtimes in the .mypy_cache directory are no longer a problem, as we now use a hash of the source file.)

  5. timabbott commented on Aug 2, 2017

    @timabbott
    timabbott
    on Aug 2, 2017
    Author

    I think @hackerkid saw this earlier today in the Zulip project; but we may be a bit behind 0.521 (haven't updated since PyCon).

  6. hackerkid commented on Aug 3, 2017

    @hackerkid
    hackerkid
    on Aug 3, 2017

    I got the error after I switched from a branch invite-link which had the model MultiUserInvitation to the branch get-user-2g which didn't had the model. But interestingly I was not able to reproduce the error when I switched from invite-link to master. I tried the switching multiple times and was able to obtain the same behavior. All the branches were rebased on top of upstream/master.

    During each switch I did

    • ./tools/provision
    • ./tools/run-mypy

    I have attached the tar of my Zulip directory including the cache here.

    Update: fixed the get-user-2g-link. Thanks ethanhs

  7. emmatyping commented on Aug 3, 2017

    @emmatyping
    emmatyping
    on Aug 3, 2017
    Member

    (the correct get-user-2g link)

    @hackerkid did this happen consistently? Could you try updating the version of Mypy that you use? It appears your version is old.

  8. gvanrossum commented on Aug 25, 2017

    @gvanrossum
    gvanrossum
    on Aug 25, 2017
    Member

    I downloaded the tarball but could not repro the issue with mypy 0.521, so I think it's not worth looking into further. Let us know whether you still see this with master.

  9. timabbott commented on Aug 25, 2017

    @timabbott
    timabbott
    on Aug 25, 2017
    Author

    Hmm, I've definitely seen this since 0.521. I just upgraded to the latest mypy master in case that helps; will let you know when I next see it (it's about once a week, usually after a series of switches between branches including one that deletes things).

  10. gvanrossum commented on Aug 25, 2017

    @gvanrossum
    gvanrossum
    on Aug 25, 2017
    Member

    If and when you see it, tarring up your entire zulip tree (.git and all) like @hackerkid did would be very helpful.

  11. gvanrossum commented on Sep 13, 2017

    @gvanrossum
    gvanrossum
    on Sep 13, 2017
    Member

    We've seen a similar crash internally, where the assert referenced a class that was deleted by the change. Sadly it was also impossible to repro. Which means that at least the work-around of deleting the cache shouldn't be too painful.

  12. gvanrossum commented on Sep 26, 2017

    @gvanrossum
    gvanrossum
    on Sep 26, 2017
    Member

    An internal user claims this can be reproduced as follows:

    1. Start with a clean slate
    2. Run mypy
    3. Make an edit
    4. Run it again, with -i
    5. Revert the edit (e.g. git stash)
    6. Run it again, with -i -- BOOM

    The edit in step 2 could be a whitespace edit, as long as it's a type-checked file.

  13. ilevkivskyi commented on Oct 2, 2017

    @ilevkivskyi
    ilevkivskyi
    on Oct 2, 2017
    Member

    @gvanrossum I now have a simple repro for this crash inspired by #4043. Here is the recipe:

    Initial setup:

    # a.py
from b import x

# b.py
from c import x

# c.py
x = 1

    with any previous mypy caches removed. Steps to reproduce:

    • run mypy a.py
    • delete (or comment out) definition of x in c.py
    • run mypy -i a.py (it will show an obvious error)
    • add random whitespace to a.py
    • run mypy -i a.py again -- AssertionError: Cannot find x for c.x

  14. gvanrossum commented on Oct 2, 2017

    @gvanrossum
    gvanrossum
    on Oct 2, 2017
    Member

    Thanks! Adding an unused variable definition to a.py in the 4th step will work too. I've also confirmed that this crash was introduced by the same PR (#2014) that caused #4043. I haven't played with fixes yet.

  15. gvanrossum
    closed this as completedin #4045on Oct 4, 2017
  16. gvanrossum
    added a commit that references this issue on Oct 4, 2017

    Delete cache for a module if errors are found (#4045)

    51fb765
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    crashtopic-incremental

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions