How can I get my assertions in pytest to stop being abbreviated with ellipsis?

Question

How can I get my assertions in pytest to stop being abbreviated with ellipsis ?

Every time I have an assertion that reports error it truncates all my strings with "..." why is that? how can I stop that?

Arouse

assert("a"*(10**6) == "a"*(10**6 - 1)). Why do you not want truncation? – Dorri

obviously i know why the values are truncated and the risk of not truncating. but it's greatly outweighing the fact that i have no idea what the test failure is when an assert is done between two larger strings. – Arouse

See a simple hack here: #19172054 – Nee

Answer 1

The other solution doesn't work in general. In our use case, for example, pytest truncates parameters passed to functions called from assertion statements:

E                   AssertionError: assert None is not None
E                    +  where None = search('\\n\\*\\s.*\\b[Tt]upla item 0\\b',
'@beartyped pep_hinted() parameter pep_hinted_param=((<function <lambda> at
0x7fe278838830>,),) violates PEP type hint...7fe278838830>,):\n  * Not int, bool, or
float.\n  * Tuple item 0 value <function <lambda> at 0x7fe278838830> not str.')

beartype_test/unit/pep/p484/test_p484.py:124: AssertionError

Note the ellipses (i.e., ...) that pytest interjected without our permission between the substrings violates PEP type hint and 7fe278838830>,):\n in the above output, replacing a mission-critical substring needed to make sense of that output.

Thanks, overly helpful pytest developers. You shouldn't have. ^Really.

what's the problem, bruh?

To corroborate both @NicholasDiPiazza and @MatthiasUrlichs, Pytest sadly ignores -v options when truncating parameters passed to functions called from assertion statements – which is more than mildly infuriating.

Pytest developers, if we ask for verbosity four friggin' times, it means: "Yes, we really do want verbosity, are well-apprised of the potential pitfalls, and could care less because uncensored error reporting is much more important than minimizing those nebulous theoretical harms that frankly don't apply to real-world continuous integration."

what's the solution, bruh?

Our working solution is as follows:

• Pass the --showlocals option to the addopts setting in the project's top-level pytest.ini file: e.g.,

addopts = -vvvv --showlocals -p no:xvfb -r a --doctest-glob=

• Assign the verbose string (or object representation, or whatever) being truncated to a local variable in the problematic test. In our use case, for example, we assign the verbose exception message being truncated to a local variable in that test: e.g.,

def test_problem_function():
    with raises(Exception) as exception_info:
        raise ValueError("a"*1024)

    exception_str = str(exception_info.value)   # <--- This is where the magic happens.
    assert problem_function("a"*1024, exception_str)

def problem_function(substr, bigstr):
    return substr not in bigstr

• Manually inspect pytest output for that variable's value instead: e.g.,

========================================================== FAILURES ===========================================================
______________________________________________ test_truncated_exception_message _______________________________________________

    def test_truncated_exception_message():
        with raises(Exception) as exception_info:
            raise ValueError("a"*1024)
    
        exception_str = str(exception_info.value)
>       assert problem_function("a"*1024, exception_str)
E       AssertionError: assert False
E        +  where False = problem_function(('a' * 1024), 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

exception_info = <ExceptionInfo ValueError('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa') tblen=1>
exception_str = 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaa'

beartype_test/unit/pep/p484/test_p484.py:39: AssertionError

While the AssertionError itself remains truncated, the --showlocals option correctly preserves the untruncated value of the local exception_str test variable. Voilà!

This actually works as advertised, with the caveat that this only works as advertised for strings (or object representations, or whatever) that are directly assignable to local variables in tests. That's typically but not always the case – which is why pytest really needs to start respecting -vvvvvvvvvvvvvv options in its error reporting.

Enchantment

The negative, emotional tone hinders the answer being easily understandable – Judkins

Answer 2

Could you show an example? This is how the output looks for me:

    def test_foo():
>       assert "a"*100 == "a"*100 + "b"
E       assert 'aaaaaaaaaaaa...aaaaaaaaaaaaa' == 'aaaaaaaaaaaaa...aaaaaaaaaaaab'
E         Skipping 89 identical leading characters in diff, use -v to show
E         - aaaaaaaaaaa
E         + aaaaaaaaaaab
E         ?            +

I think the diff makes it perfectly clear what's failing - but with -v (as the message suggests) I get the full diff:

    def test_foo():
>       assert "a"*100 == "a"*100 + "b"
E       assert 'aaaaaaaaaaaa...aaaaaaaaaaaaa' == 'aaaaaaaaaaaaa...aaaaaaaaaaaab'
E         - aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
E         + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab
E         ?  

Chunk

the answer must be the "-v" that you are adding. i'll look if we are adding that – Arouse

Can you try with -vvvv and see what you get? that's what we use. – Arouse

Same output with -vvvv. – Chunk

i was going to go into the pytest code and find the abbrevation logic but i then decided to not care – Arouse

I'd still appreciate simply seeing an example of an assertion where -v doesn't work, so I can at least open an issue, or fix it (I'm one of pytest's core developers) – Chunk

Just had encountered this myself; my case was comparing if an object is in a list was of custom objects (in my case mock.Call objects). Unpacking calls into tuples made ellipsis go away in -v mode. – Chiropody

I'm seeing this too. Arguments are -sxvvvv. async_fn = functools.partial(functools.partial(<function TestQos.test_basic_qos_default_args at 0x7f3e11f7ba60>, <bound method Te...t_basic.TestQos object at 0x7f3e13c019e8>>, amqp=<trio_amqp.tests.testcase.ProxyAmqpProtocol object at 0x7f3e1395c0f0>) @NicholasDiPiazza – Parks

This doesn't actually work. To corroborate both @NicholasDiPiazza and @MatthiasUrlichs, Pytest sadly ignores -v options when truncating AssertionError output – which is more than mildly infuriating. Pytest developers, if I ask for verbosity four friggin' times, it means: "Yes, I really do want verbosity, am well-apprised of the potential pitfalls, and could care less because complete error reporting is vastly more important than minimizing those nebulous theoretical harms that frankly don't apply to real-world continuous integration." – Enchantment

Answer 3

for me just adding below was sufficient.

-vvvv

good practice is to have your test runner script accept addition @

where you can pass additional flags such as above or -k some_test_substring as needed

Micrometer