Is there an idiomatic way to install systemd units with setuptools?

Question

I'm distributing a module which can be imported and used as a library. It also comes with an executable—installed via console_scripts—for people to use.

That executable can also be started as a systemd service (Type=simple) to provide a daemon to the system.

systemd services need to refer to absolute paths in their ExecStart lines, so I need to figure out where the console_script got installed to.

With other build systemd like CMake, autotools or Meson, I can directly access the prefix and then use something like configure_file (meson) to substitute @PREFIX@ in foo.service.in, and then install the generated unit to $prefix/lib/systemd/system/.

I just can't figure out how to do this with setuptools. Inside setup.py, sys.prefix is set to python's prefix, which isn't the same thing. I have tried overriding install_data and install, but these approaches all have various problems like breaking when building a wheel via pip, or leaving the file around when uninstalling. (I'd be OK just not installing the service when we're not being installed system-wide; systemd wouldn't be able to find it if the prefix is not /usr or /usr/local anyway.)

Someone pointed me at a website which says that you're not supposed to do this with setuptools:

Note, by the way, that this encapsulation of data files means that you can't actually install data files to some arbitrary location on a user's machine; this is a feature, not a bug.

Which sounds like I'm just going against the grain here. I'm really looking for pointers or examples to tell me what I'm supposed to be doing here. If that's "don't use setuptools", fine - but then advice on what the preferred thing is would be welcome.

Speechless

https://mcmap.net/q/1166571/-apt-dependies-for-pypi-package – Motorboat

@sinoroc Distribution packages invoke an upstream build system to cause everything to be laid out in the correct way. Are you basically telling me that I should roll the build system myself instead of using something that the Python community provides? In that case I will write a Meson build file, but I'd be re-inventing logic that e.g. setuptools already knows, and I'd lose pip installability for the module (the unit is optional), so it feels a bit sad. – Speechless

I don't think you need to install data files to an arbitrary location on the machine. You can just do systemctl enable path/to/my.service. I've done this many times when deploying to embedded systems but I didn't use setuptools. – Hemo

Answer 1

I would say don't use setuptools for this, it's not what it's made for. Instead use the package manager for the target distribution (apt, yum, dnf, pacman, etc.).

I believe systemd and such things that are specific to the operating system (Linux distribution, Linux init system) are out of scope for Python packaging, pip, setuptools, etc. Keep as much as reasonably possible under setuptools's responsibility, so that the project stays pip-installable, and so that the project itself provides easy way to get access to systemd specific files but don't install them (maybe with a command my-thing-generate-systemd-files --target path/to/output for example, that would simplify the task for the users of the project). And on the other side provide Linux-distribution-specific packages (deb, rpm, etc.), if I am not mistaken there are tools that allow to build such packages starting from the Python project (a sdist or the source code repository).

Some ideas (quick search without testing, not recommendations):

• https://pypi.org/project/pyp2rpm/ (from Fedora)
• https://src.fedoraproject.org/rpms/pyproject-rpm-macros (from Fedora)
• https://pypi.org/project/stdeb/ (unmaintained?)
• https://pypi.org/project/py2deb/ (unmaintained?)
• https://snapcraft.io/docs/python-apps (from Ubuntu, new standard?)
• https://fpm.readthedocs.io/
• https://briefcase.readthedocs.io/en/stable/reference/platforms/linux/appimage.html
• etc.

Imaginable

Mmm, if I could sensibly wrap the setuptools distribution to add the platform / system specific stuff on top that would make sense. It would then know what the prefix is to be able to generate the unit and call setup.py with the same prefix. I wonder what a nice way to do that is. Thanks for the answer! – Speechless

Answer 2

I can't fully answer your question, because I don't know for sure how to access the absolute path to the console_scripts executables. That said, you mention that cmake and meson both provide access to the installation prefix, but it seems like you might not know that both can also make pip-installable python packages. That might be one way to do what you want, so I wanted to briefly outline that option for you.

Setuptools is one example of what's called a "build backend". In order to install a package from source code, pip needs a way to convert the code into a standard format (called a wheel file). This is what a build backend does. Here's a brief overview of some common backends, and what each is good for:

• setuptools: packages with a handful of standalone extension modules (i.e. python modules written in languages other than python)
• flit: packages written entirely in python
• cmake, meson: python bindings for a larger project already being built with the corresponding system, typically written in C/C++

I'm assuming that your project is written entirely in python. Using the cmake or meson build backends in that case would be somewhat of an awkward fit, because both tools are really meant for doing lots of compiling and linking, and you wouldn't need any of that. But both tools are capable of handling pure python libraries, and should also give you access to the prefix you need to make your systemd files.

Here's a link to a tutorial on the meson build backend. Note that it describes how to build an extension module, since that's what the backend is meant for, but hopefully you can figure out how to adapt it to your project. The basic idea is to make a normal meson configuration file, then to specify the meson build backend in pyproject.toml. I don't know off-hand how to access the console_scripts install prefix. I don't even know for sure if such access is possible, but I do think this could be a worthwhile approach to consider.

Strata