Given the name of a Python package, what is the name of the module to import? [duplicate]

Question

Does somebody know the logic behind Python modules names vs the name of the actual package used in easy_install?

A few (amongst others) example that seem a bit unlogical to me:

• We do easy_install mysql-python, but the import is in fact import MySQLdb
• We do easy_install python-memcached, but the import is in fact import memcache (without the trailing d)

I didn't find a consistent way of finding the equivalence. For some modules, it took me a lot of browsing to find it. What am I doing wrong?

Braithwaite

It is up to the package maintainer how to name the package and modules inside it. – Arianism

Answer 1

Regrettably, there's no method to the madness. The name in the package index is independent of the module name you import. Disastrously some packages share module names. If you install both, your application will break with even odds. (Ruby has this problem too)

---

Packaging in Python is generally dire. The root cause is that the language ships without a package manager. Ruby and Nodejs ship with full-featured package managers Gem and Npm, and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them. Nodejs arrived 2009 and already has 14k packages. The venerable Python package index lists 24k. Ruby Gems lists 44k packages.

Fortunately, there is one decent package manager for Python, called Pip. Pip is inspired by Ruby's Gem, but lacks some vital features (eg. listing packages, and upgrading en masse). Ironically, Pip itself is complicated to install. Installation on the popular 64-bit Windows demands building and installing two packages from source. This is a big ask for anyone new to programming.

Python's devs are ignorant of all this frustration because they are seasoned programmers comfortable building from source, and they use Linux distributions with packaged Python modules.

Until Python ships with a package manager, thousands of developers will needlessly waste time reinventing the wheel.

---

Python 3 solves many problems with packaging. There aren't any packages for Python 3.

Stocky

I sure hope this answer is partially tongue-in-cheek. I keep telling myself parts of this can't be serious, but other parts are spot on... – Tambour

Is installing pip on win64 really that bad? I usually just download the setuptools installer then run easy_install pip. Though I am using 32bit python, so maybe that's where the difference lies. – Miniskirt

That's right. There's no setuptools installer for 64 bit Python. "Currently, the provided .exe installer does not support 64-bit versions of Python for Windows, due to a distutils installer compatibility issue". I discourage friends from installing 64 bit Python. – Stocky

pip can list packages: installed packages with pip freeze and online packages (stored on PyPI) with pip search. – Saez

This answer is outdated. But python dependency is still a mess. – Heredia

“reinventing the wheel” Haha, I see what you did there! – Nag

"There aren't any packages for Python 3" ... huh? – Alfi

It seems worse now, pip search is deprecated and they forward you to a web-search gui... – Neogaea

Answer 2

Unlike the accepted answer suggests, it is actually possible, although indeed quite cumbersome.

The module name(s) can be found top_level.txt file in the metadata directory of a package installation.

To access it with python (replace python-memcached with your package name):

>>> import pkg_resources as pkg # included in setuptools package
>>> metadata_dir = pkg.get_distribution('python-memcached').egg_info
>>> open('%s/%s' % (metadata_dir, 'top_level.txt')).read().rstrip()
'memcache'

Or with an equivalent bash "one liner":

cat $(python -c "import pkg_resources; \
print(pkg_resources.get_distribution('python-memcached').egg_info)")/top_level.txt

Keep in mind that some packages install multiple modules, so the method I'm presenting can return multiple module names.

Fiske

I would add open('%s/%s' % (metadata_dir, 'top_level.txt')).read().rstrip().split('\n') which returns a list of one or more package names. As the answer stands it returns a string with a single package name, or multiple package names separated by a \n. Very useful answer though! – Placer

It looks like top_level.txt is not always present in all packages. stackoverflow.com/questions/48598805/… I just encountered this situation with scipy. – Placer

Even worse, sometimes the content of top_level.txt is just wrong – Alfi