Packaging resources with setuptools/distribute

Question

I'm developing an Python egg that has several .txt dependencies (they're templates used to generate files by the egg itself), and I'm struggling to get those dependencies copied to site-packages during setup.py install. According to the distribute documentation...

Filesystem of my package:

setup.py
package
|--- __init__.py
|--- main.py
|--- binary (calls main.py with pkg_resources.load_entry_point)
|--- templates
     |--file1.txt
     |--file2.txt

In setup.py:

setup(
    [...]
    eager_resources = ['templates/file1.txt', 'templates/file2.txt']
)

Within my package:

from pkg_resources import resource_string
tpl = resource_string(__name__, 'templates/file1.txt')

...this combination of configuration and filesystem should result in file1.txt and file2.txt being available through pkg_resources.resource_string. Unfortunately, they're not being copied to site-packages during setup.py install. What am I missing?

Thanks!

Audriaaudrie

Answer 1

The information can be found in the setuptools documentation for including package data: https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files

Basically, you just need to set include_package_data=True in your setup.py file. If you are using subversion or CVS, all versioned files will be included. If not, you can specify which files to include with a MANIFEST.in file.

I believe distribute supports this as well.

You can then access the files as you would without them being packaged. i.e. in main.py you could have:

import os.path
f = open(os.path.join(os.path.dirname(__file__),'templates','file1.txt'))
print f.read()
f.close()

and this would work in the packaged version as well. One caveat is that you will have to also set zip_safe = False in setup.py so that all the files are unzipped during installation.

Purvey

Excellent, thank you for the help. Would you mind clarifying the difference between "data files" and "resources", as defined by the setuptools documentation you linked to? I was following the instructions for resources, but they appear to be qualitatively different. Thanks! – Bortz

Well, your question was the first time I've ever come across "resources" so I too am curious what they are for. But data files are definitely the way to go for stuff like templates. – Purvey

Note that you need not use zip_safe=False. Instead, use pkg_resources.resource_stream('packagename', 'templates/file1.txt').read() and setuptools will get the stream for you from the filesystem or from the zipped egg. – Wo

The documentation specifically recommends against using __file__ type access for packaged data (peak.telecommunity.com/DevCenter/…). Use the Resource Management API like the OP was already using, and like @JasonR.Coombs suggested. – Purify

This solution only works when using a manifest file or a version control system. Is there a way to do this using only the package_data (or eager_resources) arguments of setup? – Broderick