python/c++ - Compiling shared library with cmake and installing with distutils

Question

I have a boost.python project that I compile using cmake and make. It's part of a python module, and I want to be able to install that module using distutils. I have followed the instructions here to create a CMakeLists.txt file that first compiles the shared library, then sets up setup.py so that make install with install the python module. However, whilst all the python files are recognised by distutils and moved to the build directory, the shared library isn't, and I really don't know why.

My project directory structure:

• project
  • build (python distutils directory)
  • doc (module documentation)
  • module (main module directory)
    • core (dir for the boost project/library
      • CMakeLists.txt - builds shared library
    • other_py_files (several directories of pure python files)
  • CMakeLists.txt
  • setup.py.in
  • setup.py (generated by cmake)

My setup.py.in file:

from distutils.core import setup
setup(
    name='module',
    version='${PACKAGE_VERSION}',
    packages=['module', 'module.core', 'module.other_py_files'],
    package_dir={'': '${CMAKE_CURRENT_SOURCE_DIR}'},
)

My CMakeLists.txt:

cmake_minimum_required (VERSION 2.6)
project (module)

add_subdirectory(module/core)

find_program(PYTHON "python")

if (PYTHON)
  set(SETUP_PY_IN "${CMAKE_CURRENT_SOURCE_DIR}/setup.py.in")
  set(SETUP_PY "${CMAKE_CURRENT_BINARY_DIR}/setup.py")
  set(DEPS "${CMAKE_CURRENT_SOURCE_DIR}/pyQCD/__init__.py")
  set(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/build/timestamp")

  configure_file(${SETUP_PY_IN} ${SETUP_PY})

  add_custom_command(OUTPUT ${OUTPUT}
    COMMAND ${PYTHON} ${SETUP_PY} build
    COMMAND ${CMAKE_COMMAND} -E touch ${OUTPUT}
    DEPENDS ${DEPS})

  add_custom_target(target ALL DEPENDS ${OUTPUT})

  install(CODE "execute_process(COMMAND ${PYTHON} ${SETUP_PY} install)")

endif()

I thought distutils was supposed to add shared library extensions automatically to build directory, or have I got that wrong somewhere? (The shared library is an importable python module built on the C api (boost.python), so it should work right?)

Ardellardella

Here is a link to a C/Python module that uses CMake - gitlab.com/ideasman42/blender-mathutils - distutils is just used to extract Python prefix. (although there is a distutils setup.py too, that's not used for the CMake build) – Justice

Note that you should specify the WORKING_DIRECTORY when you call COMMAND ${PYTHON} ${SETUP_PY} build. It should point to where your setup.py file is. If it is not specified, you might get an empty package. – Exhilarate

Answer 1

In the end I followed the answer here and overrode the build_ext command. As I want the build to be cross platform, and since the shared library lies inside the module source tree, I import the module shared library add use the file property to get the path to the shared library from setup.py.

Ardellardella

Answer 2

scikit-build is a custom implemenation of setup that replaces distutils.core.Extension and uses cmake to compile the sources. A variety of sample projects can be found here. In the case you're using Pybind11, there is another simple example of usage here.

Argueta