Why does "python setup.py sdist" create unwanted "PROJECT-egg.info" in project root directory?

Question

When I run

  python setup.py sdist

it creates an sdist in my ./dist directory. This includes a "PROJECT-egg.info" file in the zip inside my "dist" folder, which I don't use, but it doesn't hurt me, so I just ignore it.

My question is why does it also create a "PROJECT-egg.info" folder in my project root directory? Can I make it stop creating this? If not, can I just delete it immediately after creating the sdist?

I'm using the 'setup' function imported from setuptools. WindowsXP, Python2.7, Setuptools 0.6c11, Distribute 0.6.14.

My setup config looks like:

{'author': 'Jonathan Hartley',
 'author_email': '[email protected]',
 'classifiers': ['Development Status :: 1 - Planning',
                 'Intended Audience :: Developers',
                 'License :: OSI Approved :: BSD License',
                 'Operating System :: Microsoft :: Windows',
                 'Programming Language :: Python :: 2.7'],
 'console': [{'script': 'demo.py'}],
 'data_files': [('Microsoft.VC90.CRT',
                 ['..\\lib\\Microsoft.VC90.CRT\\Microsoft.VC90.CRT.manifest',
                  '..\\lib\\Microsoft.VC90.CRT\\msvcr90.dll'])],
 'description': 'Utilities for games and OpenGL graphics, built around Pyglet.\n',
 'keywords': '',
 'license': 'BSD',
 'long_description': "blah blah blah",
 'name': 'pygpen',
 'options': {'py2exe': {'ascii': True,
                        'bundle_files': 1,
                        'dist_dir': 'dist/pygpen-0.1-windows',
                        'dll_excludes': [],
                        'excludes': ['_imaging_gif',
                                     '_scproxy',
                                     'clr',
                                     'dummy.Process',
                                     'email',
                                     'email.base64mime',
                                     'email.utils',
                                     'email.Utils',
                                     'ICCProfile',
                                     'Image',
                                     'IronPythonConsole',
                                     'modes.editingmodes',
                                     'startup',
                                     'System',
                                     'System.Windows.Forms.Clipboard',
                                     '_hashlib',
                                     '_imaging',
                                     '_multiprocessing',
                                     '_ssl',
                                     '_socket',
                                     'bz2',
                                     'pyexpat',
                                     'pyreadline',
                                     'select',
                                     'win32api',
                                     'win32pipe',
                                     'calendar',
                                     'cookielib',
                                     'difflib',
                                     'doctest',
                                     'locale',
                                     'optparse',
                                     'pdb',
                                     'pickle',
                                     'pyglet.window.xlib',
                                     'pyglet.window.carbon',
                                     'pyglet.window.carbon.constants',
                                     'pyglet.window.carbon.types',
                                     'subprocess',
                                     'tarfile',
                                     'threading',
                                     'unittest',
                                     'urllib',
                                     'urllib2',
                                     'win32con',
                                     'zipfile'],
                        'optimize': 2}},
 'packages': ['pygpen'],
 'scripts': ['demo.py'],
 'url': 'http://code.google.com/p/edpath/',
 'version': '0.1',
 'zipfile': None}

Benbow

FTR: Related question (but not duplicate: this one is about egg-info file created during sdist, the other about egg-info created during installation): https://mcmap.net/q/281293/-why-is-an-egg-info-file-created-when-i-39-m-using-distutils/821378 – Succentor

Answer 1

This directory is created intentionally as part of the build process for a source distribution. A little gander at the developer guide for setuptools gives you a hint as to why:

But, be sure to ignore any part of the distutils documentation that deals with MANIFEST or how it's generated from MANIFEST.in; setuptools shields you from these issues and doesn't work the same way in any case. Unlike the distutils, setuptools regenerates the source distribution manifest file every time you build a source distribution, and it builds it inside the project's .egg-info directory, out of the way of your main project directory. You therefore need not worry about whether it is up-to-date or not.

You may safely delete the directory after your build has completed.

Bonus edit:

I customize the clean command within my setup.py on many of my Python projects to delete *.egg-info, dist, build, and *.pyc and other files. Here's an example of how it's done in setup.py:

import os
from setuptools import setup, Command

class CleanCommand(Command):
    """Custom clean command to tidy up the project root."""
    user_options = []
    def initialize_options(self):
        pass
    def finalize_options(self):
        pass
    def run(self):
        os.system('rm -vrf ./build ./dist ./*.pyc ./*.tgz ./*.egg-info')

# Further down when you call setup()
setup(
    # ... Other setup options
    cmdclass={
        'clean': CleanCommand,
    }
)

To illustrate, after running python setup.py build on a dummy project called "poop" (Yes, I'm very mature), this happens:

$ python setup.py build
running build
running build_py
creating build
creating build/lib
creating build/lib/poop
copying poop/__init__.py -> build/lib/poop

And now if we run python setup.py clean:

$ python setup.py clean
running clean
removed `./build/lib/poop/__init__.py'
removed directory: `./build/lib/poop'
removed directory: `./build/lib'
removed directory: `./build'

Tada!

Diaphaneity

Many thanks for the insight. I could never figure out whether I should be scouring the docs for distutils or setuptools or distribute. I'm going to hang on a while before selecting your answer, just in case there is something like a kwarg I haven't spotted yet to tell setuptools not to do this. – Benbow

So I added lines to my Makefile to delete *.egg-info on 'make clean', or after doing a 'make sdist'. Thanks for reassuring me this won't necessarily break everything. – Benbow

You're welcome. That's exactly what I do as well from within my setup.py. I'm glad I was able to help! :) – Diaphaneity

@Diaphaneity How did you add this extra behavior to your setup.py so that, for example, a python setup.py clean removes the .egg-INFO directory? – Mourn

Take a look at this setup.py to see how I do it. It's pretty easy. – Diaphaneity

Running build and clean as a one-liner: python setup.py build && python setup.py clean – Veliz

Answer 2

The -egg.info folder isn't always a temporary artifact you can delete.

For example, if you use pip install -e YOURPACKAGE for an "editable" install (works via symlink like python setup.py develop so you don't have to re-install a package every time you edit it locally), the -egg.info folder is required at runtime when your package is imported in another source. If it doesn't exist, you will get a DistributionNotFound error.

Kleon

I upvoted sometime earlier, but now I'm realizing this answer does not clarify: does this apply when packaging, or only when installing? We could reasonably assume that the editable install can generate its own egg info folder from the sdist during install. Meanwhile, the question is more focused on the egg info folder created when building the sdist. – Rachael

Running pip install -e . and deleting the egg.info gives no error at all. – Howarth

Answer 3

Note that you can have the PROJECT.egg-info artifacts disappear completely from your sdist.

The command setup.py egg_info will use the source root as the egg base by default, resulting in the PROJECT.egg-info directory being packaged into the sdist.

You can configure the egg base by passing the option --egg-base. This will create the PROJECT.egg-info directory somewhere else, leaving it out of your source distribution completely. You might also use a setup.cfg to set that property.

The following command to create a sdist without a PROJECT.egg-info works for me:

python setup.py egg_info --egg-base /tmp sdist

Or in a setup.cfg:

[egg_info]
egg_base = /tmp

Oiler

Sadly, setuptools intentionally broke support for absolute egg_base pathnames at some point. The following fatal error is now emitted: setup() arguments must *always* be /-separated paths relative to the setup.py directory, *never* absolute paths. Sigh. – Joycelynjoye

Answer 4

Pythons packaging and build system is broken imho. So there are many hacks and workarounds for things that one would belive work out of the box.

However, the "cleanest" hack I found for deleting the *.egg-info is using the normal clean --all switch along with the egg_info to place the *.egg-info file in a subfolder that will be cleaned by the clean command. Here an example:

In your setup.cfg use something like this:

[egg_info]
egg_base = ./build/lib

where ./build/lib is a folder that clean --all will delete. Then when building your project with setuptools use the clean command with the --all flag, e.g.

python setup.py bdist_wheel clean --all

if you want to build a source bundle as well just make sure to build bdist_wheel before sdist so the build/lib folder exists, e.g.:

python setup.py bdist_wheel sdist clean --all

Rodomontade

Answer 5

An alternative solution to jathanism's method could be the usage of the egg_info hook. I use it to clean up before each build:

from pathlib import Path
from setuptools import setup
from setuptools.command.egg_info import egg_info

here = Path(__file__).parent.resolve()

class CleanEggInfo(egg_info):
    def run(self):
       shutil.rmtree(here.joinpath('build'), ignore_errors=True)
       shutil.rmtree(here.joinpath('dist'), ignore_errors=True)

        for d in here.glob('*.egg-info'):
            shutil.rmtree(d, ignore_errors=True)

        egg_info.run(self)

setup(
    cmdclass={
        'egg_info': CleanEggInfo,
    }
)

Lophophore