Install package with separate source directory in editable mode

Question

Situation

This is the structure of an example package:

$ tree Foo/
Foo/
├── setup.py
└── src
    ├── bar.py
    └── __init__.py

The package's name shall be foo however the package's source files are placed in the src folder.

The files' contents are:

setup.py:

from setuptools import setup

setup(
    name='foo',
    version='1',
    packages=['foo'],
    package_dir={'foo': 'src'}
)

__init__.py:

from .bar import bar

print(bar)

bar.py:

bar = 1

Problem

When doing pip install Foo everything is fine and I can use the package as I'd expect:

>>> import foo
1

However when I install the package in editable mode (pip install -e Foo) it doesn't work:

>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'foo'

Instead I can import src now:

>>> import src
1

Installing with the -e switch created a site-packages/foo.egg-link file with the following content:

<path-to>/Foo
.

A corresponding foo.egg-info directory has been created in Foo:

$ tree Foo/foo.egg-info/
Foo/foo.egg-info/
├── dependency_links.txt
├── PKG-INFO
├── SOURCES.txt
└── top_level.txt

The SOURCES.txt file contains:

$ cat Foo/foo.egg-info/PKG-INFO
foo.egg-info/PKG-INFO
foo.egg-info/SOURCES.txt
foo.egg-info/dependency_links.txt
foo.egg-info/top_level.txt
src/__init__.py
src/bar.py

and the top_level.txt:

$ cat Foo/foo.egg-info/top_level.txt 
foo

So during the import Python seems to ignore the information which is presented in Foo/foo.egg-info/ but instead just scans Foo/ for a package called foo (which is not found).

Question

How can I prepare a package which uses a separate source folder (whose name is different from the package name) so that I can install it in editable mode and still use it as it was installed normally (that is import <package-name> should work in any case for example).

Caniff

Out of curiosity, why is the package directory named src in the first place? – Erastianism

@Erastianism Admittedly I normally do follow the convention to name the source root directory after the package (so foo in this example) which technically resolves the above mentioned problem. However I discovered that distutils / setuptools support custom names for the source directory and so I wondered how well this plays together with installing in editable mode. Thus this question is more of "curious relevance". – Caniff

Answer 1

I believe that in the src folder there should be a foo folder that contains init.py and bar.py that way the package name will be foo and you can import the foo package. Not sure why it behaves differently in editable mode though.

Sapers

Answer 2

I was facing the same problem with a src-style layout as described here.

Here it is noted what pip editable install does:

This is functionally the same as setuptools’ develop mode, and that’s precisely the mechanism used for setuptools-based projects.

So to answer the question, setting the root of the package (an empty string) with package_dir={'': 'src', ....} is what worked for me. Then, src has been added to the path in the ....egg-link file.

Salem