Deleting py.test tmpdir directory after successful test case

Question

I'm running tests that create a lot of large files during execution. For this purpose I would like to delete the tmpdir directory if a test has passed. But if a test fails the contents of tmpdir should be left unchanged.

I know how to determine the result of the test:

from _pytest.runner import runtestprotocol

def pytest_runtest_protocol(item, nextitem):
    reports = runtestprotocol(item, nextitem=nextitem)
    for report in reports:
        if report.when == 'call':
            # value will be set to passed or filed
            test_outcome = report.outcome
            # But what next?

return True

but I don't know how to find out the path of the created tmpdir directory.

Cradle

If you are running Python 3, you have the option to use a temporary direcory as a context manager with with. See the example at docs.python.org/3.5/library/tempfile.html for details. You might create something based on that implementation. – Veneer

FWIW pytest's tmpdir fixture should keep 3 directories around (no matter if the test failed or not) and delete older ones. – Longawa

The way its working for me: pytest is keeping 4 "base" tempdir directories. Each base dir contains all of the separate dirs for the tests. – Cradle

Answer 1

you can retrieve easily your tmpdir from funcargs of your actual item.

In your case:

from _pytest.runner import runtestprotocol

def pytest_runtest_protocol(item, nextitem):
    reports = runtestprotocol(item, nextitem=nextitem)
    for report in reports:
        if report.when == 'call':
            # value will be set to passed or filed
            test_outcome = report.outcome
            # depending on test_outcome value, remove tmpdir
            if test_outcome is "OK for you":
               if 'tmpdir' in item.funcargs:
                  tmpdir = item.funcargs['tmpdir'] #retrieve tmpdir
                  if tmpdir.check(): tmpdir.remove()

return True

For the story, item.funcargs is a dictionary containing {arguments:value} passed to the test item we are currently checking. So the first step is to check that tmpdir is indeed an arg of the actual test, then retrieved it. And finaly check its existence before removing it.

Hope this will help.

Edit: it seems like your pytest_runtest_protocol(..) has not still fully initialized the item. To make sure it is..

Just override the pytest_runtest_teardown(item), it acts on each test item once its run is done (succesfully or failed). Try to add the method like that:

def pytest_runtest_teardown(item):
   if item.rep_call.passed:
      if 'tmpdir' in item.funcargs:
         tmpdir = item.funcargs['tmpdir'] #retrieve tmpdir
         if tmpdir.check(): tmpdir.remove()

And of couse, dont forget the following (given in the documentation) to have access to your reports easily.

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call,):
    # execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()
    # set an report attribute for each phase of a call, which can
    # be "setup", "call", "teardown"
    setattr(item, "rep_" + rep.when, rep)

Richardo

I've been trying to do this the same way but without success. I changed the "OK for you" to 'passed'. After running a simple test I get a internal error "INTERNALERROR> if 'tmpdir' in item.funcargs: INTERNALERROR> TypeError: argument of type 'NoneType' is not iterable". The problem is that "item.funcargs is empty. Any other suggestions? – Cradle

I edited just in case your pytest_runtest_protocol(..) hasn't yet fully initialize the test item you are processing. – Richardo

Answer 2

You should create a tmpdir fixture that creates the tempdir, passes it into your code and afterwards deletes it.

Additionally, the fixture must be set to always delete the tempdir, even on failure. Otherwise you may leave behind an unclean state, which could cause other tests to fail (without the user noticing).

Instead I recommend either

1. Using --pdb to drop into Python Debugger on errors. The fixture will not yet have cleaned up and you can inspect the files.
2. Creating a custom option that allows you to disable cleanup of the tmpdir.
3. Creating a custom tmpdir fixture that copies all tmpfiles to a user-configurable place (again, using a custom option) and cleans up the tmpdir afterwards.

In any case an unclean tmpdir state will be a conscious decision by the user and will prevent unexpected sideeffects.

Birthright

Could you elaborate on "Additionally, the fixture must be set to always delete the tempdir, even on failure. Otherwise you may leave behind an unclean state, which could cause other tests to fail (without the user noticing)."? Since every tmpdir is its own subdir, I don't see how that could happen. – Longawa

Yes, I'm aware that I have to create a fixture that will delete the content in the finalizer. The trick is how to check to outcome of the test? – Cradle