Doctests: How to suppress/ignore output?

Question

The doctest of the following (nonsense) Python module fails:

"""
>>> L = []
>>> if True:
...    append_to(L) # XXX
>>> L
[1]
"""

def append_to(L):
    L.append(1)
    class A(object):
        pass
    return A()

import doctest; doctest.testmod()

This is because the output after the line marked XXX is <__main__.A object at ...> (which is returned by append_to). Of course, I could put this output directly after the line marked XXX but in my case this would distract the reader from what shall be actually tested, namely the side effect of the function append_to. So how can I suppress that output or how can I ignore it. I tried it with:

"""
>>> L = []
>>> if True:
...    append_to(L) # doctest: +ELLIPSIS
    ...
>>> L
[1]
"""

def append_to(L):
    L.append(1)
    class A(object):
        pass
    return A()

import doctest; doctest.testmod()

However, this yields a ValueError: line 4 of the docstring for __main__ has inconsistent leading whitespace: ' ...'.

What I don't want to do is to change the line append_to(L) to something like _ = append_to(L) which would suppress the output, because the doctest is for documentation purposes and to show the reader how the module is supposed to be used. (In the case being documented, append_to should be used statement-like and not like a function. Writing _ = append_to(L) would deviate the reader from this.)

Amphora

Answer 1

rewrite: This actually does work now; I realized that the "doctest" I had written earlier was actually not being parsed as the module docstring, so the test wasn't passing: it was just not being run.

I made sure to double-check this one.

__doc__ = """
>>> L = []
>>> if True:
...    append_to(L) # doctest: +IGNORE_RESULT
>>> L
[1]
""".replace('+IGNORE_RESULT', '+ELLIPSIS\n<...>')

def append_to(L):
    L.append(1)
    class A(object):
        pass
    return A()

I'm not sure if this qualifies as more readable or not. Note that there's nothing special about <...>: it will only work if the actual return value has that form, as it does in this case (i.e. it's <module.A object at 0x...>). The ELLIPSIS option makes ... "match any substring in the actual output" ¹. So I don't think there's a way to get it to match the entirety of the output.

update: To do this the "proper" way, it looks like you'd want to call doctest.register_optionflag('IGNORE_RESULT'), subclass doctest.OptionChecker, and arrange for an instance of that subclass to be used by the doctest. Presumably this means that running your doctest via $ python -m doctest your_module.py is not an option.

Moncear

Would be nice the have an answer with the "proper" way with register_optionflag, implemented subclass of OutputChecker and how to run the test. Or at least have a link to an example that does something similar. – Mulciber

Answer 2

I ended up in this question because I need the +IGNORE_RESULT behavior but for running doc tests on sphinx documentation. The replace answer posted by @intuited does not work in that case and there was no code for the mentioned "proper" solution. Therefore I post what I ended up with.

As a direct answer to the question the solution would be:

__doc__ = """
>>> L = []
>>> if True:
...    append_to(L) # doctest: +IGNORE_RESULT
>>> L
[1]
"""

def append_to(L):
    L.append(1)
    class A(object):
        pass
    return A()

if __name__ == "__main__":
    import doctest

    IGNORE_RESULT = doctest.register_optionflag('IGNORE_RESULT')

    OutputChecker = doctest.OutputChecker
    class CustomOutputChecker(OutputChecker):
        def check_output(self, want, got, optionflags):
            if IGNORE_RESULT & optionflags:
                return True
            return OutputChecker.check_output(self, want, got, optionflags)

    doctest.OutputChecker = CustomOutputChecker
    doctest.testmod()

For my particular need of doc testing sphinx documentation add to the conf.py file:

import doctest

IGNORE_RESULT = doctest.register_optionflag('IGNORE_RESULT')

OutputChecker = doctest.OutputChecker
class CustomOutputChecker(OutputChecker):
    def check_output(self, want, got, optionflags):
        if IGNORE_RESULT & optionflags:
            return True
        return OutputChecker.check_output(self, want, got, optionflags)

doctest.OutputChecker = CustomOutputChecker

Then test with a command such as:

sphinx-build -M doctest source_dir build_dir source_dir/file.rst

Mulciber

Answer 3

Please try to give fully self-contained, runnable code; even when you're demonstrating a problem, the code should run on its own to reproduce the problem, so solutions can copy the code directly for demonstrating the answer.

I don't know of a clean solution to this, and I've hit it before; it seems a side-effect of the fuzzy (more bluntly: sloppy) test definitions doctests provide. A workaround is to remember that you can define functions within doctests, so you can contain a whole test as a single function rather than as its individual statements.

def append_to(l):
    """
    >>> L = []
    >>> def test():
    ...     if True:
    ...         append_to(L) # XXX
    >>> test()
    >>> L
    [1]

    >>> def test():
    ...     L = []
    ...     if True:
    ...         append_to(L) # XXX
    ...     return L
    >>> test()
    [1]

    """
    l.append(1)
    return object()

if __name__ == "__main__":
    import doctest
    doctest.testmod()

Autobiographical

What do you mean by giving self-contained, runnable code? The code I gave can be run directly with the Python interpreter (producing exactly the failures I mentioned) - at least this works here using Python 2.6. As to your answer: I have also thought of wrapping everything in a function so that the return value will be None, but exactly as you have written, that doesn't look like a very clean solution. – Amphora

I just miscopied it; the doctest was written as a separate string instead of a property of what's being tested--I never do that. Anyway, the above is reasonably clean so long as your test really does resolve to a single result. If a test doesn't fit that pattern, eg. if you have multiple interconnected statements, testing some result values and not others, then it won't work very well. There really needs to be a doctest.DISCARD flag. – Autobiographical