makepyfile with a test method + inline_run disables assertion rewriting (?)

[belugame]

OS: Ubuntu 16.04, python2.7, pytest 4.3.1

I'm working on a plugin for pytest, therefore I create temporary python files which hold test methods. This however makes the assertion-error report of pytest less human-readable.

pytest_plugins = ["pytester"]
def test_foo(testdir):                                                              
    some_test = """                                                                 
    def test_bar(): pass                                                            
    """                                                                             
    testdir.makepyfile(some_test)                                                   
    testdir.inline_run()                                                            
    assert "abcdefghijk" == "abdcdefghijkl" 

Gives me:

_____________________ test_foo ___________________

testdir = <Testdir local('/tmp/pytest-of-foo/pytest-159/test_foo0')>

    def test_foo(testdir):
        some_test = """
        def test_foo(): pass
        """
        testdir.makepyfile(some_test)
        testdir.inline_run()
>       assert "abcdefghijk" == "abdcdefghijkl"
E       AssertionError: assert 'abcdefghijk' == 'abdcdefghijkl'

When I rename def test_bar to just def bar, I get the expected easy to read error:

>       assert "abcdefghijk" == "abdcdefghijkl"
E       AssertionError: assert 'abcdefghijk' == 'abdcdefghijkl'
E         - abcdefghijk
E         + abdcdefghijkl
E         ?   +         +

So it seems it does not like the test-method in the file. Not sure if this is a bug, or I am using it wrong?

pip list

Package        Version
-------------- -------
atomicwrites   1.3.0  
attrs          19.1.0 
funcsigs       1.0.2  
more-itertools 5.0.0  
pathlib2       2.3.3  
pip            19.0.3 
pluggy         0.9.0  
py             1.8.0  
pytest         4.3.1  
scandir        1.10.0 
setuptools     40.8.0 
six            1.12.0 
wheel          0.33.1 

[gqmelo]

I bumped into the same issue when testing a plugin and asserting on a big list.
The failure message is truncated and I can't see what the difference is.

I could reproduce this with a very minimal example:

def test_plugin(request, testdir):
    testdir.makepyfile("def test_foo(): assert True")
    # This assertion would print a nice diff
    # assert [1, 2] == [1]
    testdir.runpytest()
    # This one just prints a generic AssertionError
    assert [1, 2] == [1]

I changed pytest's assertrepr_compare function to just raise an exception at the beginning. When doing that, the first assertion above raises the exception. The second one doesn't. This shows that somehow the assertion rewrite is being undone after testdir.runpytest().

On my real test a workaround for now is to use difflib in the failure message:

    assert obtained_content == expected_content, '\n'.join(difflib.Differ().compare(obtained_content, expected_content))

It is not clean, but at least I get a similar diff when it fails.

[blueyed]

I think this is fixed by #6646.