How to package a library for cibuildwheel/auditwheel?
Asked Answered
Q

0

6

I'm puzzled by an issue with Rtree, a Python package that I occasionally contribute to. The package uses cibuildwheel to generate binary wheels for PyPI. This package uses ctypes to bind to the C API for libspatialindex.

Here is a high-level description of the process:

  1. The wheel building is triggered using GitHub Actions with build.yml
  2. A local build of libspatialindex is compiled with install_libspatialindex.bash, and the *.so* shared libraries are copied into the Rtree Python package (specifically rtree/lib)
  3. The wheel building is controlled by setup.py, which copies rtree/lib files to the wheel. (Probably package-data would be easier, but that's off-topic). This initial binary wheel is built e.g. python -m build --wheel which adds rtree/lib/libspatialindex.so and rtree/lib/libspatialindex_c.so.
  4. auditwheel "repairs" the binary wheel by magically finding the Linux binaries (not sure how this is done), then using patchelf to generate a new Rtree.libs/libspatialindex-91fc2909.so.6.1.1 shared library for the binary wheel.

The end result is a Linux binary wheel with two copies of the shared library (one in Rtree.libs and the original in rtree/lib). The preferred library is the one created by auditwheel, in Rtree.libs.

A workaround could be to remove rtree/lib, with the help of wheel unpack and wheel pack.


Question

How should cibuildwheel/auditwheel be configured to "discover" and use a external library to be used with ctypes? (For instance, the description above is the "bundle binaries in package" approach). This challenge is described by one of the auditwheel limitations that it cannot statically determine the library dependencies, since the code dynamically loads libraries at runtime using ctypes.

Pointers to modern practices would be much appreciated.

Quirita answered 25/7, 2023 at 21:32 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.