Python: How do I find which pip package a library belongs to?

Question

I got a script transferred from someone else. And there is a module imported into the script. I'm wondering what is the best way to find out which pip package installed this library (other than search online).

I tried to import the package and then just do help() on it but didn't got much information. Is there a reliable and pythonic way to achieve this?

For example:

In the script it has a line

from impala.dbapi import connect

Without searching on internet, how can I find out that following package can install this library? as you can see in this case the package name is is different from the name used in pip.

pip install impyla

Review

module.__file__ will give you the file path. Perhaps that will indicate the proper package name. It may also have a module.__package__ you can check. – Arneson

Why do you need to be able to do this without searching the Internet? That's the obvious solution. – Dredi

@IanAuld has the right approach. Are you sure pip knows about this package? Does it appear if you do pip search ? – Copperplate

@Dredi In an ideal word every python package would have a comprehensive online documentation. But of course we don't live in an ideal world, and some packages may not have sufficient online documentation to discern any name differences. – Pharmacist

Does this answer your question? How to find the package name for a specific module? – Seisin

module.__file__ and module.__package__ still refer to the import package name, not the distribution package name. There is no trace of the distribution package name in site-packages where the pip packages are installed. – Chaoan

If you were able to use the help command on the module created by this code, that implies that you succeeded in importing it, which implies that its own import succeeded, which implies that you already have the necessary package installed. – Exscind

Answer 1

Short answer: You can't.

The core of the reason is that the name to import the package and the name to install the package come from different namespaces. When you run the import command, Python is looking for the package in the local environment. When you tell pip to install a package, it's looking for something to install from PyPI (or somewhere else if you told pip to look elsewhere).

It would make sense for these two names to always be the same, but there's no guarantee. Installation is a matter of choosing which set of files to download and install, while importing is a matter of choosing what installed files to run, but the names for those files do not have to stay the same during the installation process. And that's how we get confusion like pip install impyla, import impala.

If you had access to the setup.py file for the package, you could look in there for the name (if you look at the GitHub for impyla/impala, you'll see a name='impyla', line inside the call to setup), but if the package was installed via pip, you won't have the setup.py file locally, so this option is pretty much right out.

It's not a great state of affairs. There's simply no guarantee that you can find the PyPI name for a package from just having local access to the package code and the "real" import name for the package. That said, if you're unfamiliar with the package anyway, you're probably going to want to look up documentation and more info on the internet anyway. Just one more thing to look up, I guess.

Sokul

Answer 2

If you do not have the package installed, you'll have to use a pip search or a poetry search (or a Google search).

In the case you have the package already installed (i.e. you can import it) , you can get the package name with importlib.metadata:

• In Python >=3.8, you can use the standard library importlib.metadata
• For Python <3.8, there is importlib_metadata (link to documentation). Replace importlib.metadata with importlib_metadata in the examples below.

Example on usage:

>>> from importlib.metadata import packages_distributions

>>> packages_distributions()

 'asttokens': ['asttokens'],
 'backcall': ['backcall'],
 'bitarray': ['bitarray'],
 'colorama': ['colorama'],
 'decorator': ['decorator'],
 'executing': ['executing'],
 'importlib_metadata': ['importlib-metadata'],
 'impala': ['impyla'],
 'IPython': ['ipython'],
 'jedi': ['jedi'],
 'matplotlib_inline': ['matplotlib-inline'],
 'parso': ['parso'],
 'pickleshare': ['pickleshare'],
 'pip': ['pip'],
 'prompt_toolkit': ['prompt-toolkit'],
 'pure_eval': ['pure-eval'],
 'puresasl': ['pure-sasl'],
 'pygments': ['Pygments'],
 '_distutils_hack': ['setuptools'],
 'pkg_resources': ['setuptools'],
 'setuptools': ['setuptools'],
 'six': ['six'],
 'stack_data': ['stack-data'],
 'thrift': ['thrift'],
 'thrift_sasl': ['thrift-sasl'],
 'traitlets': ['traitlets'],
 'wcwidth': ['wcwidth'],
 'zipp': ['zipp']}

In this case, you would look for the "impala" entry in the packages_distributions() dictionary:

>>> packages_distributions()['impala']
['impyla']

You may also check for example the version of the package:

>>> from importlib.metadata import version

>>> # Note: Using the name of the package, not the "import name"
>>> version('impyla')
'0.18.0'

Bonus: Checking the name of the module from a function or class name

>>> connect.__module__.split('.')[0]
'impala'

---

Note that this does not guarantee that a package was installed from PyPI. You could, if you want, create your own package called "impyla", "matplotlib", whatsoever, install it, and use it as you wish.

Acknowledgment

Answer 3

Ask your friend to run this code at the end of his scripts to see what address it belongs on his machine, consequently finding name of the package from its address.

import name_of_the_module
name_of_the_module.__file__

However, and this is a very interesting point, you should notice sometimes data scientists use a simple .py in their directory to summarize their codes for later calculations. I had the same problem with import train in my test codes on an old linux machine which wouldn't work on a new one. After getting its address by running above code, finally I understood it is just a simple train.py file in my directory which helps test data to easily import dataset for testing neural network model without visualizing sorting&importing methods mentioned in training section again.

Your final solution, and this is an absolute solution, is to pack all of the original libraries and extract them onto new machine. See this answer for using conda-pack to pack packages and check also my comment about it if you are using anaconda environment on the new machine.

Ouachita