Cmake: using conan pybind11 package

Question

I'm having trouble understanding how to use pybind11 conan package. I can use some others, but pybind11 is giving me hard time.

My starting point is as follows:

conanfile.txt:

[requires]
pybind11/2.7.1

[generators]
cmake

main.cpp:

#include <pybind11/pybind11.h>

int add(int i, int j) {return i + j;}

PYBIND11_MODULE(cobind, m) {m.def("add", &add);}

CMakeLists.txt

cmake_minimum_required(VERSION 3.21)
project(cobind11 VERSION 1.0 LANGUAGES CXX)

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()

pybind11_add_module(cobind main.cpp)

The log of my suffering trying to make this work:

1. Starting point

If I try to build as above the project I get the following error:

CMake Error at CMakeLists.txt:10 (pybind11_add_module):
  Unknown CMake command "pybind11_add_module".

-- Configuring incomplete, errors occurred!

2. Adding find_package(pybind11 REQUIRED)

If I add a line find_package(pybind11 REQUIRED) I get the following error:

CMake Error at CMakeLists.txt:16 (find_package):
  By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "pybind11",
  but CMake did not find one.

3. Adding include([...]/pybind11Tools.cmake)

If I add a line #include(${CONAN_PYBIND11_ROOT}/lib/cmake/pybind11/pybind11Tools.cmake) I get the following error:

CMake Error at /home/[...]/pybind11Tools.cmake:100 (set_property):
  set_property could not find TARGET pybind11::pybind11.  Perhaps it has not yet been created.

like in this issue https://github.com/pybind/pybind11/issues/3388. Maybe it is fixed in the new release?

4. Upgrading to 2.8.1

The fresh pybind11 is 2.8.1, let's upgrade

pybind11/2.8.1: Not found in local cache, looking in remotes...
pybind11/2.8.1: Trying with 'conancenter'...
ERROR: Unable to find 'pybind11/2.8.1' in remotes

OK. One can read between the lines, that it used to work, so maybe let's downgrade?

5. Downgrading to 2.4.3

If I require pybind/2.4.3 instead of pybind/2.7.1 in conanfile.txt I get

fatal error: Python.h: No such file or directory
  112 | #include <Python.h>
      |          ^~~~~~~~~

like in this issue https://github.com/pybind/pybind11/issues/1781. Differently from that issue, installing python*-dev does not help. But whatever, I don't want to use old pybind anyway.

6. Trying a conanfile from test package

The conancentral recipes contain a test package (https://github.com/conan-io/conan-center-index/tree/master/recipes/pybind11/all/test_package), which is tested automatically when the package is created. Let's try that instead!

CMake Error at CMakeLists.txt:13 (find_package):
  By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "pybind11",
  but CMake did not find one.

Am I just hopeless???

Maybe, but! I can:

• Build the https://github.com/pybind/cmake_example
• Build the https://github.com/pybind/python_example
• Build the conancentral pybind11 recipe! The same that fails when I extract the test package to a separate folder. Whaaat??

Sometime

Try adding find_package(pybind11 REQUIRED) before the line containing pybind11_add_module(cobind main.cpp). – Understrapper

Unfortunately it does not work with a different error (I'll paste in the post). However, in the meantime I found this PR in pybind11 github.com/pybind/pybind11/pull/3420, so it might be that things will start working in the next release. – Sometime

In the "conanbuildinfo.cmake" file generated by conan, that you are including in your CMakeLists.txt file, many variables are set for the different dependencies. Particularly, you get a "CONAN_LIB_DIRS_<dependency>" variable for each dependency with the proper library folder of that dependency. That means you can use ${CONAN_LIB_DIRS_PYBIND11}/cmake/pybind11/pybind11Tools.cmake to get the full path of that file. – Understrapper

Thanks @darcamo, but I can't make this work either. I updated my post, this whole thing is super infuriating. – Sometime

The pybind issue you posted is for multi-config generators, but you are using the cmake generator. Therefore, it is not related to your problems. – Understrapper

The most recent version of pybind available in conan is 2.7.1 (do a conan search pybind11 -r=all to check). You can create an issue (or a pull-request) here to add a new version. – Understrapper

Answer 1

I have used pybind in the past (two or three years ago) and it worked without problems with conan. However, I tried to install it now and faced similar problems. This might be related to the evolution of conan towards conan 2.0 (an alpha version was released today) and that the recipe was not updated to account changes.

An alternative that you might consider, is to install pybind with conan using a different generator. More specifically, the CMakeDeps generator. With the CMakeDeps generator conan will create a pybind11-config.cmake file for you and you just need to use find_package(pybind11 REQUIRED) in CMakeLists.txt. That is, there is no "conan specific stuff" in your CMakeLists.txt file.

conanfile.txt

[requires]
pybind11/2.7.1

[generators]
CMakeDeps

CMakeLists.txt

cmake_minimum_required(VERSION 3.21)
project(cobind11 VERSION 1.0 LANGUAGES CXX)

# Tell cmake to also search in the buld folder for the "<library>-config.cmake"
# files
list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}")
find_package(pybind11 REQUIRED)

pybind11_add_module(cobind main.cpp)

main.cpp

#include <pybind11/pybind11.h>

int add(int i, int j) { return i + j; }

PYBIND11_MODULE(cobind, m) { m.def("add", &add); }

With this I was able to build the library and use it in Python.

---

Conan deeper integration with CMake

You can also use the CMakeToolchain generator in addition to the CMakeDeps generator. It generates a conan_toolchain.cmake file that you pass to the cmake command with --toolchain conan_toolchain.cmake. If you use it, there is no need to add the list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}") line to your CMakeLists.txt file. Furthermore, the settings you specify in conan, such as build type and compiler, will impact cmake. That is, you don't need to specify these things in both conan and cmake. This seems to be where conan is going, regarding cmake integration, in the comming 2.0 release.

Understrapper

This does not work, I get exactly the same error with this approach. – Pathology

not useful as well – Fullerton