How to update git metadata in a Python Packages using Setuptools and PBR
Asked Answered
D

1

0

Working with versions generated from git repo tags by pbr

I'm having issues getting the version info from my package, which is setup as a normal Python package using setuptools with the pbr extension. pbr will pull the version info from the tags in a git repo so there is a single source of truth.

I did get the basics working - see this Q&A for more info.

It is working overall, if I make a full distribution of the package. For example: py setup.py sdist will create a full distribution and the version tag and other meta data from git is updated.

But I don't need full distributions of this package, my team is just working on it locally out of a git repo. I'm also using pyinstaller to create an executable of the main script for user distribution (without needing Python). So I want a simple, quick command that will just update the metadata, based on the latest updated to the git repo, without spending extra time for SetupTools to build and create the full distribution package, which will not be used anyway.

pbr will pull version (from tags), authors, and changelog info from the repo, so it's a big timesaver, allowing us to keep most of the metadata directly in git.

Questions

Is there anything we can do to get the package metadata to update with pbr, without making a full distribution?

Particularly on the version:

  1. When or with which commands will setup.py using the pbr extension actually go update the version from the git repo so that any command querying the version will get the updated version?
  2. Where is the version actually stored in this setup? (I can't find it...)

More info and some code

I've got a very short setup.py, for use with pbr extension:

#!/usr/bin/env python

from setuptools import setup
setup(
    setup_requires=['pbr', 'setuptools'],
    pbr=True,
)

Now, if I update the latest commit in the git repo with a tag, the command py setup.py --version will return the new updated version based on that tag, but any other direct method I can find will still return the old version string. So that command does not appear to store the new version string anywhere.

Here's a few methods I've tried from my package __init__.py:

import pkg_resources  # part of setuptools

v1 = pkg_resources.require("md2mat")[0].version
print('v1 {}'.format(v1))
v2 = pkg_resources.get_distribution('md2mat').version
print('v2 {}'.format(v2))

from pbr.version import VersionInfo

v3 = VersionInfo('md2mat').release_string()
print('v3 {}'.format(v3))

# Update per sinoroc's comment:
# As of Python 3.8, you can use this from the stdlib,
# which removes run-time dependencies on `pbr` or `setuptools`
import importlib.metadata

__version__ = importlib.metadata.version('Example')

All of the above return the old version string (last time a full distribution was made).

So, is there anything I can do short of py setup.py sdist or a similar full distribution build command (bdist, bdist_egg, etc.) to simply update my package info in place so the above methods will give me the latest version string and other git metadata when the main package script is run?

That would allow us to update the package locally as developers after checking out a new commit, and to then run pyinstaller to create our exe and release our end user version of the script.

Debus answered 24/10, 2019 at 22:13 Comment(4)
I have to mention import importlib.metadata; __version__ = importlib.metadata.version('Example') since it's in the standard library since Python 3.8. It prevents from having run time dependencies (install_requires) on setuptools or pbr.Silvan
I saw that somewhere and tried it, but it didn't work. I am only on 3.7.4... probably worth updating to try this out. I still can't get the pkg_resources methods to work with pyinstaller... thanks for the tip.Debus
There's a backport called importlib-metadata to ease with the transition, but then it's just trading a dependency for another one.Silvan
FYI, I got importlib-metadata working with Python 3.8.0, but PyInstaller wouldn't successfully compile my package, so I'm back on Python 3.7. I believe all the other information and steps are still required, so while this provides a cleaner method for accessing the metadata in scripts, it's only a nice-to-have for the moment, that I can't use at the moment.Debus
D
1

It appears that the info for version is stored in the package.egg-info/PKG-INFO file. Therefore, any setuptools command that updates the egg-info will pull in the git info.

In the PKG-INFO file, the Version is showing up for me currently in the 3rd line:

Version: 2.0.4.0b2

Updating the Version string directly in the PKG-INFO file does cause the scripts to display the updated string when accessing through pkg_resources or pbr.version.

The quickest command to have PBR automatically update the metadata in the .egg-info folder appears to be:

py setup.py egg_info

longer / more complex commands that will also update the info include build, sdist, bdist, etc. See py setup.py --help-commands for more available setup.py commands.


Authors and Changelog

While the above is true for versioning, the AUTHORS and ChangeLog files DO NOT get updated by pbr for the egg_info or build commands.

However, these files are generated during the sdist and bdist commands (including all versions of bdist, I think). So it still looks like to update all the metadata from Git one of the full distribution build commands is required.

Commands that will not generate the AUTHORS and ChangeLog files:

py setup.py egg_info
py setup.py build
py setup.py develop

(Note: I use the fairly recent py command to run python, which allows you to setup your default environment for running python, but you may need to use python or python3 instead)

Debus answered 24/10, 2019 at 22:27 Comment(2)
Doesn't it work with python3 setup.py develop? If it does, then it seems to me that it's the one you should use, as it is closer to what you actually want to do. No need to build distributions.Silvan
I tried it and develop does run egg_info but doesn't update AUTHORS or ChangeLog (unfortunately). I updated the answer. This feels like a good enhancement request for pbr...Debus

© 2022 - 2024 — McMap. All rights reserved.