How to find out the Python library installation path?

Question

For installing third-party Python packages I have used a setup.py script that uses setuptools.setup() to install a bunch of packages. After the installation I can find these packages on one machine under /usr/local/lib/python2.7/dist-packages and on another machine under /usr/lib/python2.7/site-packages.

Now I want to write a Python script that finds out where the third-party packages have been installed. How can I do that?

1) sys.prefix=sys.exec_prefix is on both machines "/usr".

2) The python executable is on both machines /usr/bin/python.

3) distutils.sysconfig.get_python_lib() is /usr/lib/python2.7/dist-packages ("local" is missing) on one machine and /usr/lib/python2.7/site-packages on the other machine.

Edmiston

Answer 1

If the packages have already been installed, you could just import them, and look into their __file__ property:

>>> import mymodule
>>> print mymodule.__file__
'/path/to/mymodule.py'

Chanterelle

Interesting trick. But it cannot be used, if the installed packages are different on the different hosts and you don't know whether a certain module is installed on a certain host. – Edmiston

This was useful for getting to my answer. – Kemper

how would this work if you cannot import the package in the first place? – Hearten

I like this solution. ObUpdate — in plaid3 it's print(mymodule.__file__) – Stylography

Answer 2

Nearly found a solution to this problem. I refereed to this question and this question so thanks to answers given there.

Firstly you'll need to get a list of all installed modules into a list. Use this question to capture the output of the solution to this question.

Now you have a list of all installed python modules. You will need to see the format of list and then format it properly to get individual elements as the names of the modules.

Then you can import the modules from their names as strings as explained here. Then as alejandro already said mymodule.__file__ contains the paths.

This is one solution that should work. Not very elegant but I am just a Python beginner who is better at google search than Python

---

I found a much easier way to find where modules are. This might be the "elegent" solution that OP was looking for.

import sys
print sys.path

From the Python docs about sys module sys.path contains

A list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default.

Kemper

I don't know it advance what are the possible directories to look at. It can be /usr or /usr/local or a private Python installation, it can be dist-packages or site-packages. How does setuptools know where to install to? – Edmiston

@Edmiston Updated the answer. Have a look. – Kemper

and for python3 same as above, just print(sys.path) – Whitebeam

Answer 3

A quick command line way of doing this is as follows:

% python3 -c "import sys; print(list(filter(lambda s: s.find('packages') > -1, sys.path)))"

This will list all packages folders for python3. For example:

['/home/toor/.local/lib/python3.7/site-packages', '/home/toor/.pyenv/versions/3.7.9/lib/python3.7/site-packages']

If looking or the executable path:

% python3 -c "import sys; print(sys.executable)"

Output:

% /home/toor/.pyenv/versions/3.7.9/bin/python3

Adamec

Answer 4

I found my installation path via sys.path:

my_name = "mypackage"
def _post_install():
     def find_module_path():
         for p in sys.path:
             if os.path.isdir(p):
                 for f in os.listdir(p):
                     if f == my_name:
                         return os.path.join(p, f)
     install_path = find_module_path()
     # do something we the install_path

Note: This works after the package has been registered by the setuptools, and you probably also have to handle if the package has been installed in multiple directories within the sys.path.

Mapp

Answer 5

I don't know of a good way to do this in Python (other than crawling the directories from a given root to look for a file using os.walk). If you do not get any better answers for how to do this in Python you could execute a command on your OS from python to execute a standard Linux/Unix find command on the library you are looking for. You could use the subprocess module to do this.

Hanky

I would prefer a more "elegant" solution than searching directory trees. How does setuptools know where to install to? – Edmiston

The point is that even if you could use the same method to establish the default directory this would not allow for scenarios where the user has not used the default. I do not know of any other way, but will be interested to see what other suggestions come up. – Hanky