How to make pytest run doctests as well as normal tests directory?

Question

We currently have pytest with the coverage plugin running over our tests in a tests directory.

What's the simplest way to also run doctests extracted from our main code? --doctest-modules doesn't work (probably since it just runs doctests from tests). Note that we want to include doctests in the same process (and not simply run a separate invocation of py.test) because we want to account for doctest in code coverage.

Trite

Answer 1

Now it is implemented :-).

To use, either run py.test --doctest-modules command, or set your configuration with pytest.ini:

$ cat pytest.ini
# content of pytest.ini
[pytest]
addopts = --doctest-modules

Man page: PyTest: doctest integration for modules and test files.

Dolly

that doesn't work for us, where we want to specify that tests are in mypackage/tests and doctests are e.g. in mypackage/foo. What does work is pytest mypackage/tests --doctest-modules mypackage/foo. How would one define that in pytest.ini? I've tried to express this there, but no cigar. – Accipiter

@PierreD thanks for sharing your workaround. Didn't have such file structure. Few ideas: once you have the working command-line I would try to move it to pytest.ini, like addopts = --doctest-modules mypackage/foo and see if it works. I would play with `--doctest-glob="foo/*.txt" to try to catch your files. If you would provide a git-repo with the filestructure + requirements.txt, and the result you want to have, one can fix it for your case much faster, I hope. – Dolly

well, my comment was wrong; it does what I want but for the wrong reasons: both tests and doctests are collected from both pathes. I was able to further tweak the pytest invocation in our Makefile, but it doesn't quite do what I want (we also have integration tests and integration-lite tests). Bottom-line, what I would need is a way to tell pytest to collect doctests and the regular test collection from potentially two different sets of directories. For example: doctests on everything (./mypackage), and tests only from the non-integration tests (./mypackage/tests). – Accipiter

Answer 2

This is how I integrate doctest in a pytest test file:

import doctest
from mylib import mymodule

def test_something():
   """some regular pytest"""
   foo = mymodule.MyClass()
   assert foo.does_something() is True

def test_docstring():
   doctest_results = doctest.testmod(m=mymodule)
   assert doctest_results.failed == 0

pytest will fail if doctest fails and the terminal will show you the doctest report.

Donoho

Answer 3

Could you try with the repo version of pytest and paste a session log? I'd think --doctest-modules should pick up any .py files.

Decile

Answer 4

worked with doctest as well as with plain tests in one module. for a non-doctest test to be picked up, standard py.test discovery mechanism applies: a module name with test prefix, test function with test prefix.

Unswear

Answer 5

Looks old question but put my answer here just in case.

• pytest only run tests in /*test*.py AFAIK and you need to customze with this behavior by the configurations, testpaths and python_files to run doctest in your python code files. seealso: https://docs.pytest.org/en/latest/example/pythoncollection.html#changing-naming-conventions and so on.
• pytest does not run doctest by default and you have to give the option --doctest-modules when running pytest. seealso: https://docs.pytest.org/en/stable/doctest.html

Curagh