I'm puzzled by an issue with Rtree, a Python package that I occasionally contribute to. The package uses cibuildwheel to generate binary wheels for PyPI. This package uses ctypes to bind to the C API for libspatialindex.
Here is a high-level description of the process:
- The wheel building is triggered using GitHub Actions with
build.yml
- A local build of libspatialindex is compiled with
install_libspatialindex.bash
, and the*.so*
shared libraries are copied into the Rtree Python package (specificallyrtree/lib
) - The wheel building is controlled by
setup.py
, which copiesrtree/lib
files to the wheel. (Probablypackage-data
would be easier, but that's off-topic). This initial binary wheel is built e.g.python -m build --wheel
which addsrtree/lib/libspatialindex.so
andrtree/lib/libspatialindex_c.so
. - auditwheel "repairs" the binary wheel by magically finding the Linux binaries (not sure how this is done), then using
patchelf
to generate a newRtree.libs/libspatialindex-91fc2909.so.6.1.1
shared library for the binary wheel.
The end result is a Linux binary wheel with two copies of the shared library (one in Rtree.libs
and the original in rtree/lib
). The preferred library is the one created by auditwheel, in Rtree.libs
.
A workaround could be to remove rtree/lib
, with the help of wheel unpack
and wheel pack
.
Question
How should cibuildwheel/auditwheel be configured to "discover" and use a external library to be used with ctypes? (For instance, the description above is the "bundle binaries in package" approach). This challenge is described by one of the auditwheel limitations that it cannot statically determine the library dependencies, since the code dynamically loads libraries at runtime using ctypes.
Pointers to modern practices would be much appreciated.