Debugging modifications of sys.path

Question

Some library seems to modify my sys.path, although I don't want ìt to be changed.

How can I find the python code line which alters sys.path?

Related

• Where does sys.path get replaced?

Adrianneadriano

You can first check sys.path then import module and check it again. – Baum

Answer 1

One of the first things imported is the sitecustomize and usercustomize modules; you could replace sys.path with a custom list implementation that records all changes being made.

First, find where to place a usercustomize or sitecustomize module; the site module can tell you where to place the first:

python -m site --user-site

If that directory doesn't exist yet, create it and in it put a usercustomize.py with:

import sys

class VerboseSysPath(list):
    def croak(self, action, args):
        frame = sys._getframe(2)
        print('sys.path.{}{} from {}:{}'.format(
            action, args, frame.f_code.co_filename, frame.f_lineno))

    def insert(self, *args):
        self.croak('insert', args)
        return super().insert(*args)
    
    def append(self, *args):
        self.croak('append', args)
        return super().append(*args)

    def extend(self, *args):
        self.croak('extend', args)
        return super().extend(*args)

    def pop(self, *args):
        self.croak('pop', args)
        return super().pop(*args)

    def remove(self, *args):
        self.croak('remove', args)
        return super().remove(*args)

    def __delitem__(self, *args):
        self.croak('__delitem__', args)
        return super().__delitem__(*args)

    def __setitem__(self, *args):
        self.croak('__setitem__', args)
        return super().__setitem__(*args)

sys.path = VerboseSysPath(sys.path)

This now will complain about all attempts at altering the sys.path list.

Demo, with the above placed in either the site-packages/sitecustomize.py or `python -m site --user-site`/usercustomize.py modules:

$ cat test.py 
import sys

sys.path.append('')
$ bin/python test.py 
sys.path.append('',) from test.py:3

Palaeontology

For anyone facing a similar problem: I couldn't trace the unexplained entries in sys.path by adding this code to sitecustomize.py, so i've ended up temporarily hacking this into site.py itself and managed to solve the problem (.pth file leftover from a buggy installation). site.py is located in lib folder. Notably; if you're using a virtual environment, the core library does not come with, just lib/site-packages. You can find where your base installation is by looking at the first few entries in sys.path. – Thiourea

@MarcinKonowalczyk: Ah, yes, .pth files are processed before the *customize modules are imported. Note that .pth files in the per-user site-packages directory are loaded before those in the site directory, and any line that starts with import in such a file is executed, so you could use a .pth file instead of a usercustomize module to load the same sys.path instrumentation, and so not have to alter site.py. – Palaeontology

Answer 2

Starting python with python -S causes python not to load site.py, and so its default value is preserved from when python first starts up.

Mcewen