How to temporarily modify sys.path in Python?

Question

In one of my testing scripts in Python I use this pattern several times:

sys.path.insert(0, "somedir")
mod =  __import__(mymod)
sys.path.pop(0)

Is there a more concise way to temporarily modify the search path?

Boldface

Note: sys.path.pop(0) works only if mymod doesn't prepend another path without removing it. And if that happened pop would remove the wrong path. – Chare

Answer 1

You could use a simple context manager:

import sys

class add_path():
    def __init__(self, path):
        self.path = path

    def __enter__(self):
        sys.path.insert(0, self.path)

    def __exit__(self, exc_type, exc_value, traceback):
        try:
            sys.path.remove(self.path)
        except ValueError:
            pass

Then to import a module you can do:

with add_path('/path/to/dir'):
    mod = __import__('mymodule')

On exit from the body of the with statement sys.path will be restored to the original state. If you only use the module within that block you might also want to delete its reference from sys.modules:

del sys.modules['mymodule']

Givens

This is a nice solution. Is it better to remove the path as you don't know the code within the context manager's context is not going to modify sys.path itself? – Winnick

Yes, it may be better in such cases. Just use sys.path.remove(self.path) instead of sys.path.pop(0). – Givens

to add to @dementedhedgehog 's suggestion: you might also want to catch a ValueError raised by sys.path.remove in case the code in the context has already removed the path itself – Stakeout

Answer 2

Appending a value to sys.path only modifies it temporarily, i.e for that session only.

Permanent modifications are done by changing PYTHONPATH and the default installation directory.

So, if by temporary you meant for current session only then your approach is okay, but you can remove the pop part if somedir is not hiding any important modules that is expected to be found in in PYTHONPATH ,current directory or default installation directory.

http://docs.python.org/2/tutorial/modules.html#the-module-search-path

Ieper

I cannot think of a different way, however as already mentioned, the variable sys.path will only keep its modified value within this Python session. Normally, there is no need to remove a path from sys.path, at least I cannot think of a good reason to do so. A really permanent modification of sys.path would require that you modify the PYTHONPATH environment variable of your shell. – Timekeeper

@JohannesP If "somedir" is hiding(masking) some module that is expected to be found in found in PYTHONPATH,etc then we can remove that entry to fix that. – Ieper

I remove the directory after import to not slow down the search (there potentially can be hundreds of paths) – Boldface

@Boldface then it's perfectly okay. – Ieper

Answer 3

Here is an alternative implementation of the contextmanager implementation from Eugene Yarmash (use contextlib and pathlib.Path-compatible):

import os
import sys
import contextlib
from typing import Iterator, Union

@contextlib.contextmanager
def add_sys_path(path: Union[str, os.PathLike]) -> Iterator[None]:
    """Temporarily add the given path to `sys.path`."""
    path = os.fspath(path)
    try:
        sys.path.insert(0, path)
        yield
    finally:
        sys.path.remove(path)


with add_sys_path('/path/to/dir'):
    mymodule = importlib.import_module('mymodule')

Polarimeter

For someone in the future that got here but can't understand what Union and Iterator are doing: search phrases "annotation" and "type hint/check" may help. – Hernando

Union is basically like 'one or the other', and since the function yields, it returns an iterator – Siliceous

Answer 4

If you're testing uses pytest, they have a great fixture that, among other traits, handles this exact case:

The monkeypatch fixture helps you to safely set/delete an attribute, dictionary item or environment variable, or to modify sys.path for importing... All modifications will be undone after the requesting test function or fixture has finished. The raising parameter determines if a KeyError or AttributeError will be raised if the target of the set/deletion operation does not exist

In describing syspath_prepend:

Use monkeypatch.syspath_prepend to modify sys.path which will also call pkg_resources.fixup_namespace_packages and importlib.invalidate_caches().

sample use:

def test_myfunc(monkeypatch):
  with monkeypatch.context() as m:
    m.syspath_prepend('my/module/path')
    mod =  __import__(mymod)
  # Out here, the context manager expires and syspath is reset

Classicism