Change pytest working directory to test case directory

Question

I have the following pytest directory structure:

system_tests/
  ├── conftest
  ├── pytest.ini
  │
  ├── suite_1/
  │    └── test_A.py
  │   
  └── suite_2/
       └── sub_suite_2a/
            └── test_B.py

When each test method runs, a number of third-party libraries/processes generate artifacts in the current working directory.

• When pytest is executed from the sub_suite folder (using CLI or IDE "play" button), the files are generated in the sub_suite folder, where I want them to be.
• However, when pytest is run from the system_tests folder to run all tests, all artifacts are created in the system_tests folder, which is not what I want.

Is there an easy way to force pytest to always use the test class folder as the working directory so I get the same results regardless of how or where I run a test from?

Obadias

This looks like the XY problem to me - why not tell the test method to generate the artifact in a directory relative to test script file? – Disappoint

Good point. The problem is the artifacts aren't generated by the test methods directly. They're generated by test framework libraries, log files from other running applications, etc. Right now they write these files to the working directory. I guess I'll need to rewrite some code to make all paths absolute, but was hoping there was a quicker fix. – Obadias

Answer 1

EDIT: Improved Solution

Using monkeypatch as suggested by @Kound removes the boilerplate code to restore the cwd. You can also enable autouse to automatically apply this fixture to all test functions. Add the following fixture to conftest.py to change the cwd for all tests:

@pytest.fixture(autouse=True)
def change_test_dir(request, monkeypatch):
    monkeypatch.chdir(request.fspath.dirname)

• request is a built-in pytest fixture
• fspath is the LocalPath to the test module being executed
• dirname is the directory of the test module

Any processes that are kicked off by the test will use the test case folder as their working directory and copy their logs, outputs, etc. there, regardless of where the test suite was executed.

Original Solution

The following function-level fixture will change to the test case directory, run the test (yield), then change back to the calling directory to avoid side-effects, as suggested by @hi2meuk:

@pytest.fixture
def change_test_dir(request):
    os.chdir(request.fspath.dirname)
    yield
    os.chdir(request.config.invocation_params.dir)

• request.config.invocation_params.dir - the folder from which pytest was executed
• request.config.rootdir - pytest root, doesn't change based on where you run pytest. Not used here, but could be useful.

Obadias

Glad to see you figured it out. Yeah, changing your working directory to the one where the test is hosted makes the most sense! – Youthful

The fixture proposed introduces side effects that will interfere with other tests, possibly in the future which will confuse the hell out of you. Use yield then cwd back to request.config.invocation_dir before returning as per the answer from @Farhan Kathawala. – Bullhead

The request.config.invocation_dir moved to request.config.invocation_params.dir in latest of pytest. docs.pytest.org/en/latest/reference/… – Welltodo

Answer 2

Instead of creating a fixture for each directory like suggested by @DV82XL you can simply use monkeypatch to achieve the same:

import pytest
from pathlib import Path

@pytest.fixture
def base_path() -> Path:
    """Get the current folder of the test"""
    return Path(__file__).parent



def test_something(base_path: Path, monkeypatch: pytest.MonkeyPatch):
    monkeypatch.chdir(base_path / "data" )
    # Do something in the data folder

Kishke

Neat trick, thanks! This adds some overhead to the test modules and methods, but the code is more readable. Is there any reason why base_path is defined in a fixture and not a module variable? – Obadias

Because I find it troublesome to import module variables. Normally I define the base_path in my conf.py and be able to use the fixture everywhere without worring about import problems. – Kishke

Answer 3

A different and, IMHO more robust approach: always reference your files by the complete path.

__file__ is an automatically declared Python variable that is the name of the current module. So in your test_B.py file, it would have the value: system_tests/suite_2/sub_suite_2a/test_B.py. Just get the parent and choose where to write your files.

from pathlib import Path
test_data_dir = Path(__file__).parent / "test_data"

Now you have all of them in the same place and can tell your version control system to ignore them.

If the code is inside a library, better use an absolute path since you don't know where it will be installed:

import os
from pathlib import Path

test_data_dir = Path(__file__).parent.absolute() / "test_data"

Fredric

Answer 4

Many options open to you to achieve this. Here are a few.

1. Write a pytest fixture to check if the current working directory is equal to the desired working directory, and if not, then move all the artifact files to the desired directory. If the artifacts you are generating are all the same type of file (e.g. *.jpg, *.png, *.gif) and you just want them to be in a different directory, then this may suffice. Something like this could work

from pathlib import Path
import shutil

@pytest.fixture
def cleanup_artifacts():
    yield None
    cwd = Path.cwd()
    desired_dir = Path.home() / 'system-tests' / 'suite-2' / 'sub_suite_2a'
    if cwd != desired_dir:
        for f in cwd.glob('*.jpg'):
            shutil.move(f, desired_dir)

And then you can add this fixture to your tests as needed.

2. You can configure the pytest rootdir to be the desired directory, since pytest uses the rootdir to store project/testrun specific info.

When you run pytest, run it as

pytest --rootdir=desired_path

See here for more info: https://docs.pytest.org/en/latest/customize.html#initialization-determining-rootdir-and-inifile

If both don't work for you, tell more about what your requirements are. Surely this can be done with pytest.

Youthful

I like the idea of having a cleanup_artifacts() fixture. I'm definitely going to use this to consolidate and package my output files at the end of a run. However, I tried changing rootdir, but it didn't have any visible effect. pytest changed all paths relative to the new rootdir, but the end result was the same. – Obadias