How to find "import name" of any package in Python?

Question

I wonder if is there any reliable and consistant way to get a Python package's "import name" / namespace. For example;

Package; django-haystack
Import name; haystack

or

Package; ipython
Import name; IPython

So far I know, PyPi doesn't store that information that I've checked with PyPiXmlRpc.

I also tried to automate to download the package, extract it and dig the .egg-info but some packages doesn't have that folder at all.

Any help will be appreciated and will be used for a good-manner gadget :)

Monadelphous

Answer 1

Note that what you call a package here is not a package but a distribution. A distribution can contain zero or more modules or packages. That means there is no one-to-one mapping of distributions to packages.

I'm not sure there is a way to detect what modules and packages will be installed by a distribution, other than actually installing it and introspect filesystem changes for newly added packages, modules and pth files.

Theressathereto

I understand. Major problem is the relation between a distribution and a package/packages as I see. But there must be an answer to my question. I will find the most reliable and consistent one and let you guyz know for sure. – Monadelphous

after a long long researching period, i end up with your answer that any Python distribution may have one or more packages such as the package itself and let's say a test package next to as a simple example. Anyways, i was able to finish my experimental prototype (pydoc.net) and hopefully will provide some sort of API to cover this question consistently.. Thanks Wichert. – Monadelphous

Answer 2

Wheels

I know this is an old question, but wheel packages have since been invented! Since a wheel is simply a zip file that gets extracted into the lib/site-packages directory, an examination of the contents of the wheel archive can give you the top level imports.

>>> import zipfile
>>> zf = zipfile.ZipFile('setuptools-35.0.2-py2.py3-none-any.whl')
>>> top_level = set([x.split('/')[0] for x in zf.namelist()])
>>> # filter out the .dist-info directory
>>> top_level = [x for x in top_level if not x.endswith('.dist-info')]
>>> top_level 
['setuptools', 'pkg_resources', 'easy_install.py']

So setuptools actually gives you three top level imports!

pip download

pip now has a download command, so you can simply run pip download setuptools (or whatever package you like) and then examine it.

Reverse look up

As of Python 3.10, there is a convenience function that allows reverse look up (given the import name, what's the package). The official The official docs are here

from importlib.metadata import packages_distributions
packages_distributions()
{'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'],  ...}

For the forward lookup you could also simply build the reverse this dictionary.

Uglify

As of 3.10, there's an easy way to do a reverse look-up, i.e. mapping import to distribution packages: docs.python.org/3/library/… – Overeat

Thanks, @Overeat ! I've updated my answer with this information. – Uglify

Answer 3

In principal, everything you need to get that information is in the setup.py that is supposed to be in every such package. That information would roughly be the union of the packages, py_modules, ext_package and ext_modules of the Distribution object. In fact, here's a little script that mocks out distutils.core.setup just for the purpose of getting that information.

import distutils.core
distutils.core._setup_stop_after = "config"
_real_setup = distutils.core.setup
def _fake_setup(*args, **kwargs):
    global dist
    dist = _real_setup(*args, **kwargs)

distutils.core.setup = _fake_setup

import sys
setup_file = sys.argv[1]
sys.argv[:] = sys.argv[1:]
import os.path
os.chdir(os.path.dirname(setup_file))

execfile(os.path.basename(setup_file))

cat = lambda *seq: sum((i for i in seq if i is not None), [])
pkgs = set(package.split('.')[0] for package
           in cat(dist.packages,
                  dist.py_modules,
                  [m.name for m in cat(dist.ext_modules)],
                  [m.name for m in cat(dist.ext_package)]))

print "\n".join(pkgs)

For many packages, this will work like a charm, but for a counterexample, see numpy, It breaks because numpy provides its own distutils, and I can see no obvious way around it.

Sheehan

I didn't have chance to try, will do as soon as possible and feedback. thanks in advance! – Monadelphous

yes it works for many package releases but as I understood it works in the modulefinder's logic that gives you all the modules had been used and you get the high-level namespace(s) out of it. Unfortunately, it gives more than one namespace if the release is using some other package such as test; check django-uni-form for example. I just modified your script a bit by the way; gist.github.com/1176645 – Monadelphous

And Python should make "import name" field required on PyPI! Can't be that painful... stucked. – Monadelphous

Seriously... I've been racking my brain for a while on how to get the distribution name from the import name. There's just not a way currently. I had hoped that the provides key in setup.pywould provide this. I had hoped to be able to query that through wiki.python.org/moin/PyPIXmlRpc, but no luck ): – Disrepair

Answer 4

My project johnnydep has this feature:

>>> from johnnydep import JohnnyDist
>>> dist = JohnnyDist("django-haystack")
>>> dist.import_names
['haystack']

Do note that a distribution may provide multiple import names:

>>> JohnnyDist("setuptools").import_names
['pkg_resources', 'setuptools']

or none at all:

>>> JohnnyDist("bs4").import_names
[]

Cajole

Answer 5

After searching lots of information, finally found the only thing which worked for me as expected. Example for python-dotenv package which has dotenv import name:

$ cat $(python -c "import pkg_resources; print(pkg_resources.get_distribution('python-dotenv').egg_info)")/top_level.txt
dotenv

(taken from this answer)

Adamite