"AssertionError: <string>" when a test is exec()ed

[progval]

Hi,

While migrating tests from unittest, I encountered the following bug:

$ cat foo.py
import unittest

class MyTestCase(unittest.TestCase):
    s = '''def testFoo(self):  assert True'''
    exec(s)
$ python3 -m pytest foo.py  
== test session starts ==
platform linux -- Python 3.7.3, pytest-5.3.4, py-1.7.0, pluggy-0.13.1
rootdir: /tmp
plugins: forked-1.1.3
collected 1 item                                                                                                                                                                                                               
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/_pytest/main.py", line 197, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/_pytest/main.py", line 271, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/_pytest/runner.py", line 83, in pytest_runtest_protocol
INTERNALERROR>     item.ihook.pytest_runtest_logstart(nodeid=item.nodeid, location=item.location)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/_pytest/compat.py", line 420, in __get__
INTERNALERROR>     value = instance.__dict__[self.func.__name__] = self.func(instance)
INTERNALERROR>   File "/home/dev-irc/.local/lib/python3.7/site-packages/_pytest/nodes.py", line 465, in location
INTERNALERROR>     assert isinstance(location[0], py.path.local), location[0]
INTERNALERROR> AssertionError: <string>

== 3 warnings in 0.01s ==

Context:

$ python3 -V
Python 3.7.3
$ pip3 list | grep pytest
pytest               5.3.4          
pytest-forked        1.1.3

The motivation for having a test in exec() is that it is in a py2/py3 conditional, to avoid a syntax error.

Fixing this bug is not critical for me as I'm going to drop this conditional anyway, but I thought you might want to know.

Thanks for developing pytest!

[blueyed]

This is fixed in master (#6517), but certainly a good test case that we should add in general.

[blueyed]

Closing. Test taken in #6576.