Is it possible to use two Python packages with the same name?

Question

I have a question about imports. The question might seem a bit contrived, but its purpose is to explore the limitations of using absolute imports for all imports in a package. PEP8 highly discourages relative imports (edit: and the Google Python Style Guide says never to use them).

Say you are given two large-ish packages that both have the same name and that both use absolute imports in accordance with PEP8:

    /pkg1
        mod1.py (contains an absolute import: 'import pkg1.mod2')
        mod2.py
        ...

    /pkg1
        mod1.py (contains an absolute import: 'import pkg1.mod3')
        mod3.py
        ...

Also say you are working on a Python project in which you'd like to use both packages. This could come up, say, if you want to use two versions of the same package in a project.

Is there a way to incorporate both packages into your project hierarchy so that you can freely use modules from both packages throughout your project?

For the solution it's acceptable to do things like use import aliases and modify sys.path temporarily. But it's not okay to change the contents of either package directory.

Cagle

Readability is important, but not as important as correctness. – Cheyney

The easiest thing to do is just rename one of the packages base name. Python doesn't really support versioned packages. – Eudosia

@Keith: is there a way to "rename one of the package's base name" without altering one of the package directories itself? Also, the packages aren't necessarily different versions. That was just an example. – Cagle

@Keith: It does, with a little help from pkg_resources in setuptools. – Cheyney

@ignacio setuptools manages versions, but I don't think it lets you use two versions at the same time. – Eudosia

@chris Well, if you don't need different simulataneous versions you can use "package namespaces" feature of setuptools (if you own both packages). If you only own one you should probably just rename yours (the directory name is the package name). – Eudosia

PEP8 is a great starting point, but not the last word; after all, "A Foolish Consistency is the Hobgoblin of Little Minds". – Micron

Answer 1

The short answer is no, Python doesn't accept two packages with the same name. (There are things called "namespace packages" that let a single package be implemented over multiple directories but they require the packages involved to be set up to cooperate with each other).

The way PEP 8 discourages explicit relative imports is one of its more questionable pieces of advice, precisely because it makes it harder to rename a package to avoid naming conflicts. If the two packages used relative imports, you could just rename one of them or nest it inside another package and be done with it.

import aliases won't help you here, because it is the name that ends up in sys.modules that matters, and that uses the name of the module as imported, rather than the name that ends up being bound in the importing module.

If you wanted to get really exotic, you could write your own importer (see PEP 302 and the 3.x importlib documentation). If you decide to go that far, you can do pretty much anything you want.

Polik

Thanks. Do you mean PEP302 instead of PEP303? – Cagle

The package name does not need to be the import name. One example is PIL and the drop-in replacement Pillow (source) – Punitive

What happens in that case? – Punitive

Answer 2

My initial tests (in Python 2.6 & 3.1) suggest the following may work:

import sys, re

import foo as foo1
for k in sys.modules:
    if re.match(r'foo(\.|$)', k):
        newk = k.replace('foo', 'foo1', 1)
        sys.modules[newk] = sys.modules[k]
        # The following may or may not be a good idea
        #sys.modules[newk].__name__ = newk
        del sys.modules[k]

sys.path.insert(0, './python')
import foo as foo2
for k in sys.modules:
    if re.match(r'foo(\.|$)', k):
        newk = k.replace('foo', 'foo2', 1)
        sys.modules[newk] = sys.modules[k]
        # The following may or may not be a good idea
        #sys.modules[newk].__name__ = newk
        del sys.modules[k]

However, I only tested this against very simple packages and only tried it as a curiosity. One problem is it probably breaks reload. Python isn't really designed to handle multiple packages with the same top-level name.

At this point, I'm tentatively going to say that it's not possible in the general case, though it's possible under certain limited circumstances but it's very brittle.

Tenancy

That will only work if you import all submodules before renaming anything in sys.modules and if the code doesn't contain any deferred imports (i.e. importing from inside a function). __name__ in all the modules will be wrong as well, as will any __module__ attributes. As your last comment noted, Python really isn't set up to support this scenario. – Polik

I'm almost prepared to say "impossible in the general case", but haven't looked enough at module internals to be sure. – Tenancy

In response to this StackOverflow question, on IRC the author of Exocet told me that his package was written to address issues like this. – Cagle

Answer 3

Actually, you should use namespaces (packages) to separate properly what modules you want to end up using. In your above code.

/pkg1
 mod1 - can just import mod2
 mod2.py
 __init__.py

/pkg2
 mod1 - can just import mod2
 mod2.py
 __init__.py

And at rest of the places you should do import pkg1.mod1 or import pkg2.mod1 as desirable.

Brooking

Just "import mod2" is an old-style ambiguous relative import and should be strongly avoided. If you want new relative imports use "from . import mod2" or use absolute imports. – Micron

It was for import within self-contained packages. How is it ambiguous? – Brooking

Answer 4

No, but you can use keyword "as" and rename the package name as you wish. EG:

from tableaudocumentapi import Workbook as table

Faubion

This is not about two packages. Your answer is about two classes with the same name. – Station

Answer 5

Paste the below line in the external init file of each package:

__path__ = __import__('pkgutil').extend_path(__path__, __name__)

For detailed help consult this repo.

Waterway