Python Setuptools and PBR - how to create a package release using the git tag as the version?

Question

How do I actually create a release/distro of a python package that uses a git repo tag for the versioning, using setuptools and pbr?

There is plenty of information on the basic setup and configuration required:

• SetupTools Documentation - setup() and setup.py configuration
• Python Packaging User Guide - Installing Packages
• PBR v3.1.1 documentation
• StackOverflow: How to use version info generated using setuptools and pbr

But where is the simple info on how to actually create the distro?

i.e. I'm looking for whatever command finds the git tag with the version info and pulls it into the configuration info, so the source with that new version info can be distributed, and the version info is discoverable from the scripts, using a method like described in this answer.

---

Additional details

I'm working on a project that will be distributed to other developers only through a git repo, not through PyPi. The project will be released to users as an executable using pyinstaller, so this package distribution will only serve a few key purposes:

1. Install/Setup the package for other developers so that dependencies/environment can be recreated cleanly.
2. Manage versioning - Current plan is to use pbr to generate versions from the Git repo tags, so those tags can be our source of truth for versioning
3. Use pbr for other auto generation of mundane items from Git, such as authors, manifest.in file, release notes, etc.

Since setuptools docs focus on setting up a fully distributable and reusable package with PyPi and pip, and pbr docs only really tell you how to modify setuptools configuration to use pbr, I can't find the info on how to just run the distribution/release process.

I'm sure it exists somewhere in the documentation, but after several false starts I'm asking here. It is implied everywhere I look that everyone either knows how to do this or it just magically happens as a part of the process.

Am I just missing the obvious?

---

Update:

Based on sinoroc's answer, it appears I need to look into development mode installs. i.e. Anyone developing the project will clone from git, and then install via using setuptools development install mode.

This wasn't directly a part of the original question, but implied, and I believe will be of interest to people in the same situation (info I couldn't easily find).

More info is available in his answer on updating some of the metadata, and via this setuptools documentation link to working in "Development Mode"

Photo

Note: I've moved onto using the Poetry package as my install/dependency management tool for my Python projects. It has a plug-in called poetry-dynamic-versioning that has the same functionality as PBR for driving version based on git tags. It is a worthy alternative project framework. – Photo

Answer 1

In short:

• python3 setup.py sdist
• python3 setup.py bdist_wheel

How do I actually create a release/distro of a python package that uses a git repo tag for the versioning, using setuptools and pbr?

The usual commands to create (source and wheel) distributions of your Python package with setuptools are: python3 setup.py sdist and python3 setup.py bdist_wheel. The distributions can then be found in the dist directory by default.

Since setuptools docs focus on setting up a fully distributable and reusable package with PyPi and pip, and pbr docs only really tell you how to modify setuptools configuration to use pbr, I can't find the info on how to just run the distribution/release process.

It is true that setuptools does not document this. It only documents the differences to distutils, and it is confusing indeed. See below for actual documentation...

But where is the simple info on how to actually create the distro?

• https://packaging.python.org/tutorials/packaging-projects/#generating-distribution-archives
• https://docs.python.org/3/distutils/sourcedist.html
• https://docs.python.org/3/distutils/builtdist.html

---

Update

Since you don't plan on publishing distributions of your project on an index such as PyPI, and you plan on using pyinstaller instead, then you can indeed most likely disregard the setuptools commands such as sdist and bdist_wheel.

Still you might want to know these commands for the development phase:

• Use commands such as python3 setup.py --version, python3 setup.py --fullname to figure out if setuptools (and in your case pbr) is catching the right info.
• Use python3 setup.py develop (or pip install --editable .) to place a pseudo link (egg-link) in your site-packages that points at your work in progress. This way your changes are always installed and importable. Important: don't use python3 setup.py install, this would copy the current version to site-packages and newer changes would not be importable.

Now I don't know how all this will work once you move on to pyinstaller. Especially since you mentioned that you want the meta info (such as the version number) to be discoverable from within your scripts. The technique with setuptools pkg_resources may or may not work in the pyinstaller context.

Rickettsia

Ah, great, very helpful - I found setup.py sdist, but without any context of what it did, beyond that I realized it updated the version based on git when used with pbr. So - if I am not distributing to PyPi, I shouldn't need bdist_wheel and the zip archive is unnecessary as well if my devs will just clone the repo? I would only be running sdist to get the file updates done locally (i.e. everyone who clones will need to run it)? – Photo

I updated the answer. You are probably looking for python3 setup.py develop, this will gather all the setuptools/pbr meta info and make it available, so that it can be accessed via pkg_resources for example. But frankly I have no idea how pyinstaller handles this. – Rickettsia

Thanks! Pyinstaller completed successfully building in pbr.version.VersionInfo calls, but blew up the exe from 6Mb to 23Mb (doesn't matter much, but slower to start as well), but when I tried using pkg_resources.require("<package>")[0].version, it didn't include all the dependencies needed and failed, but only went from 6 Mb to 7 Mb and faster. Haven't tried importing more modules directly from pkg_resources yet... – Photo

Added some info to the original question, including a link to the "Development Mode" documentation from setuptools docs. Accepted the answer. To clarify my last comment - I think pkg_resources is hiding the .version method dependency and pyinstaller is not discovering it, so if I explicitly import those functions, it will force it to discover them, and work, but figuring that out is a low priority at the moment. Thanks again! (appreciate a question upvote, if you found it worthy) – Photo

Since pkg_resources is used at run time and is part of setuptools, don't forget to add setuptools to the install_requires in your setup.py. Alternatively you could use importlib.metadata.version('Example') instead. Maybe one of these actions could help pyinstaller figure things out. – Rickettsia

I tried upgrading to v3.8.0, but pyinstaller bombs out due to some incompatibility (my best guess). I had issues getting back to a working v3.7.4, but after deleting PYTHONHOME and PYTHONPATH, and doing a clean venv with py -3.7 -m venv .venv37, then re-installing all dependencies, etc., now pyinstaller is successfully compiling using pkg_resources.get_distribution('pkg').version, when it was not working before. Something I did fixed the issue - probably deleting the environment variables, but not sure... – Photo

Okay, got pyinstaller figured out. I had to force the mypackage.egg-info directory data files (which have the package metadata) to be included when pyinstaller builds the exe. It is explained in this answer, where I laid out the different methods I found to do that. – Photo

Using py setup.py install seems to work fine to update the metadata (and it will reinstall any entry-point script), at least after having run py setup.py -e . at least once. I've made this part of my standard release process - run py setup.py install and then run pyinstaller to make the executable. If I need to run or debug local development changes, I just use my entry-point script directly (which is aptly named run.py...) – Photo

Answer 2

This is how I solved the same issue, also having read several different links. I have created a setup.py file with this content:

from setuptools import setup, find_packages

def readme():
    with open('README.rst') as f:
        return f.read()

def read_other_requirements(other_type):
    with open(other_type+'-requirements.txt') as f:
        return f.read()

setup(
      setup_requires=read_other_requirements('setup'),
      pbr=True,
      packages=find_packages('src'),
      package_dir={'': 'src'},
      include_package_data=True,
      zip_safe=True
)

I have the source code in ./src. Also, I have a setup-requirements.txt, with content:

pip==18.1
pbr==5.1.1
setuptools==40.7.0

And a setup.cfg with this content:

[metadata]
name = XXXXX
description = XXXXX
description-file = README.rst
home-page = https://github.com/XXXXX/XXXXX

So first, you install the setup-requirements:

pip install -r setup-requirements.txt

Then, whenever you have locally a commit which was tagged in GitHub, you can install it using:

python setup.py install

and it will be installed with the tagged version. You can check it by doing:

python setup.py --version

Uncouth