setup.py exclude some python files from bdist

Question

I have a django project with this kind of architecture :

• setup.py
• project/
  • __init__.py
  • manage.py
  • settings/
    • __init__.py
    • base.py
    • dev.py
  • urls/
    • __init__.py
    • base.py
    • dev.py

I wanted to deploy it in a .egg without my 'dev.py' files. I tried different ways : first, with a

find_packages(exclude=['*.dev','dev'])

, then with a MANIFEST.in which contains :

global-exclude dev.py

The second solution seems to work when I do a sdist - with this warning when I install it :

warning: no previously-included files matching 'dev.py' found anywhere in distribution 

, but does'nt work with a bdist-egg.

Here a part of my setup.py :

from setuptools import setup, find_packages
project import VERSION


packages = [
        'project',
        'project.settings',
        'project.urls',
]

setup(name='project',
  version=VERSION,
  package_dir = {'project' : 'project'},
  description  = 'My Project',
  author       = 'Simon Urli',
  author_email = '',
  url = '',
  packages = packages, #find_packages('project',exclude=['*.dev', 'dev']),
)

Note that I use python 2.6.6, maybe it matters. Any idea how to create my egg excluding the dev files properly ?

Raman

Answer 1

I had the same issue recently (although I had to build a wheel instead of an egg), the solution works the same both for bdist_egg and bdist_wheel. You have to override the method find_package_modules in build_py:

import fnmatch
from setuptools import find_packages, setup
from setuptools.command.build_py import build_py as build_py_orig


exclude = ['*.dev']


class build_py(build_py_orig):

    def find_package_modules(self, package, package_dir):
        modules = super().find_package_modules(package, package_dir)
        return [(pkg, mod, file, ) for (pkg, mod, file, ) in modules
                if not any(fnmatch.fnmatchcase(pkg + '.' + mod, pat=pattern)
                for pattern in exclude)]


setup(
    packages=find_packages(),
    cmdclass={'build_py': build_py},
)

In this example, modules named dev in all packages will be excluded from the build.

As you can see, there's no need to play with exclusions in find_packages() as you still need all packages to be included, but instead you filter the module files found in each package. The class build_py is pretty much generic and could be refactored in a separate library if you need to reuse it; the only project-specific stuff is the list of exclude patterns.

Croissant

Thanks for your answer. Since my question was 7 years ago, I'm not working on that anymore. So sorry I'm not marking it as an answer because I won't check it. If anyone confirms it works I will then ;) – Raman

Hey, no problem! I found this question while looking for an answer myself, so wanted to share my solution as the question is pretty popular; maybe it helps someone one day. – Croissant

super() doesn't work in python 2.7 and all my attempts at instantiating the parent class and getting it to work have failed. Any idea how you would do it in 2.7 by providing parameters to super()? – Disguise

@Disguise super(build_py, self).find_package_modules(...) should do. – Croissant

It didn't. I got a TypeError: __init__() takes exactly 2 arguments (3 given). After a lot of finagling i ran into the issue of NewType class vs OldType class where it expected type but got a classObj. What ultimately worked for me was build_py_orig(Distribution()).find_package_modules(package, package_dir) – Disguise

Sure, you need to subclass object to maintain compatibility to old-style classes: class build_py(build_py_orig, object): .... Otherwise, calls to super(build_py, self) will fail. – Croissant

Answer 2

def without_app(item):
    # http://docs.python.org/release/2.2.1/lib/string-methods.html
    return not bool(item.find('app_name') + 1) 

# help(filter) - use in command line to read the docstring
packages = filter(without_app, find_packages()) 

Tocharian