PytestWarning: Module already imported so cannot be rewritten: pytest_remotedata

Question

I created some unit tests and run them from the same file. For tests in the same file:

if __name__ == "__main__":
    import pytest
    pytest.main(['--tb=short', __file__])

For tests in another file:

if __name__ == '__main__':
    import pytest
    pytest.main(['./test_stuff.py', "--capture=sys"])

In either case, when I execute the file the first time, it works fine, but the second and subsequent times it gives a bunch of warnings:

============================== warnings summary ===============================
C:\Anaconda3\lib\site-packages\_pytest\config\__init__.py:754
  C:\Anaconda3\lib\site-packages\_pytest\config\__init__.py:754: PytestWarning: Module already imported so cannot be rewritten: pytest_remotedata
    self._mark_plugins_for_rewrite(hook)
  C:\Anaconda3\lib\site-packages\_pytest\config\__init__.py:754: PytestWarning: Module already imported so cannot be rewritten: pytest_openfiles
    self._mark_plugins_for_rewrite(hook)
  C:\Anaconda3\lib\site-packages\_pytest\config\__init__.py:754: PytestWarning: Module already imported so cannot be rewritten: pytest_doctestplus
    self._mark_plugins_for_rewrite(hook)
  C:\Anaconda3\lib\site-packages\_pytest\config\__init__.py:754: PytestWarning: Module already imported so cannot be rewritten: pytest_arraydiff
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
==================== 1 passed, 4 warnings in 0.06 seconds 

Is there any way to make these warnings go away?

Restarting the kernel works, but IPython's %reset and %clear aren't enough to fix it, either.

Sauder

Answer 1

Use subprocess instead of pytest.main:

if __name__ == "__main__":
    import subprocess
    subprocess.call(['pytest', '--tb=short', str(__file__)])

If the above does not print anything, try the workaround (as suggested in comments):

if __name__ == "__main__":
    from subprocess import Popen, PIPE
    with Popen(['pytest',
                '--tb=short',  # shorter traceback format
                str(__file__)], stdout=PIPE, bufsize=1,
                universal_newlines=True) as p:
        for line in p.stdout:
            print(line, end='')

Stylopodium

This doesn't print the output at all, though... :/ – Sauder

It does. :) At least in Spyder IDE with ipython console (Python 3.7). Also direct call from command line and repeated calls from inside ipython do print anything you like... Maybe you catch somewhere else your sdtout and stderr? – Stylopodium

No it doesn't print anything in Spyder. This works though: github.com/pytest-dev/pytest/issues/3143#issuecomment-464973797 – Sauder

Upvoting because your answer led me to find the other, but it would be best if yours was edited to do something similar. – Sauder

It is strange actually that my Spyder (version 3.3.3) catches and prints everything without any workarounds and yours does not. What is your spyder/python/ipython versions configuration? – Stylopodium

Spyder 3.3.3, Python 3.7.1, IPython 7.2.0, pytest 4.0.2 – Sauder

Well, my configuration differs only in pytest. I have version 4.2.0. Neverthless i edited the answer to cover the no printing case. – Stylopodium

I just tested things on Windows and subprocessed pytest really does not print output. It seems this is Windows specific problem. – Stylopodium

Answer 2

If you are only concerned about warnings, you can use --disable-warnings argument. You can also filter out only the warnings you are getting now with --pythonwarnings=PYTHONWARNINGS argument in pytest.main . pytest --help has more info about these arguments.

It seems that you are invoking pytest with ipython or jupyter. python.main imports pytest specific modules at the time of pytest initialization. When you run pytest.main again, it throws the warnings. There are two possible ways to reload pytest to avoid getting this initialization warnings.

You can either use pytest.exit after pytest.main or reload pytest after pytest.main .

Let us know if these approaches work.

Grimalkin

Be careful. While these solutions will indeed get rid of the pytest warnings, one larger issue remains: the user's to-be-tested code files are not reloaded each time, so any changes made to the code files are ignored during subsequent pytest runs. – Pseudonym

--disable-warnings works, but --pythonwarnings=PYTHONWARNINGS says INTERNALERROR> File "C:\Anaconda3\lib\warnings.py", line 236, in _getaction INTERNALERROR> raise _OptionError("invalid action: %r" % (action,)) INTERNALERROR> warnings._OptionError: invalid action: 'PYTHONWARNINGS' – Sauder

Correct syntax to hide particular warning in this case (taken from an answer here): pytest.main(['./test_stuff.py', "--capture=sys", "--pythonwarnings", "ignore:Module already imported:pytest.PytestWarning"]) You should look into reloading pytest module as that will be the cleanest solution in your case. – Grimalkin

Answer 3

It would appear that pytest just isn't really designed to perform pytest.main() calls repeatedly from within the same process. The pytest documentation mentions:

Calling pytest.main() will result in importing your tests and any modules that they import. Due to the caching mechanism of python’s import system, making subsequent calls to pytest.main() from the same process will not reflect changes to those files between the calls. For this reason, making multiple calls to pytest.main() from the same process (in order to re-run tests, for example) is not recommended.

So actually the real danger of performing multiple pytest.main() calls within the same process is that you could get false-positive or false-negative test results if you've edited your code files inbetween.

The Spyder IDE has a nifty feature though that seems to negate this issue: the User Module Reloader (UMR). If enabled, then any changed user modules are automatically reloaded when a script file is (re-)run.

Therefore I think that as long as you're working in Spyder (with the UMR feature enabled!), you can safely rerun pytest.main() without the need for a new console. The pytest module already-imported warnings you can then simply suppress, since those pytest modules will not have changed. This can be done with pytest's -W flag. Example:

if __name__ == '__main__':
    import pytest
    pytest.main(['./test_stuff.py', "--capture=sys", "-W", "ignore:Module already imported:pytest.PytestWarning"])

Pseudonym

It didn't used to have this problem. Is there a recommended way to run tests every time a file is edited and run? – Sauder

The recommended way seems to be to always use a new process for every pytest run (see github issue). – Pseudonym

@Sauder My own recommendation would be to use to an IDE that supports custom run configurations. Within the IDE set up a dedicated run configuration for your pytest tests (one that always launches a new process). – Pseudonym

I'm using Spyder, and when I configured it for "Execute in dedicated console" or "Remove all variables before execution", it doesn't help. – Sauder

@Sauder "Execute in dedicated console" results in a new console (i.e. new process) being started upon first run of the code file, but not upon subsequent reruns of the same code file. "Remove all variables" results in only user-defined variables being removed. Try using the run configuration options as described in the edited answer instead. – Pseudonym

That works, but ugh, that's a plain Python terminal that doesn't even support tab completion, let alone variable explorer, etc. :( This used to work fine, I just had tests running from my script, and every time I edit it, I Run it and it says whether the tests still pass. Maybe I should look into a different test package? – Sauder

@Sauder "evert time I edit it, I Run it and it says whether the tests still pass." This surprised me at first. I was expecting that subsequent test runs would be giving false-positive or false-negative results (due to edited code files not being reloaded). However after discovering and reading about Spyder's User Module Reloader (UMR) it's starting to make sense. This UMR ensures that previously imported user modules are automatically reloaded if they've been changed (quite a useful feature). – Pseudonym

@Sauder I think this means that as long as you're working in Spyder, you can actually safely rerun pytest.main() without the need for a new console. The warnings that you were getting you can simply suppress (see edited answer). – Pseudonym

That sounds promising, but I still get the warnings when I use your suppression switch. The --disable-warnings option in @SilentGuy's answer works though. – Sauder

@Sauder Hmm, I think the warnings filter is not working as it should due to a bug in pytest. Regarding the --disable-warnings option: do keep in mind that this will also suppress any and all warnings generated by your own code files. – Pseudonym

The -W option in very useful: "-W", "ignore::pytest.PytestAssertRewriteWarning" – Vagina

Answer 4

Subprocess is the best way to avoid this warning as describe in other answer but in case you can't use the subprocess and module is registered as pytest plugin than before calling pytest.main you can un-import the specific module using sys.modules.pop

for more details check: How to unimport a python module which is already imported?

Hoot

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Seka

A little bit of example code could be useful here - as-is, it's hard to know how this knowledge should be applied. – Badmouth