Force linking against Python release library in debug mode in Windows/Visual Studio from CMake

Question

I'm developing a Python binding for a C++ library using Boost Python, for Linux and Windows (Visual Studio).

In Windows, the static Boost Python library has a dependency against Python (this is motive for another thread, here), so, in my CMake config I need to do:

if((${CMAKE_SYSTEM_NAME} STREQUAL "Linux") OR APPLE)
     target_link_libraries(my_python_module ${Boost_LIBRARIES})
elseif(WIN32 AND MSVC)
    add_definitions(/DBOOST_PYTHON_STATIC_LIB)
    target_link_libraries(my_python_module ${Boost_LIBRARIES}) #This includes the Boost Python library
    # Even though Boost Python library is included statically, in Windows it has a dependency to the Python library.
    target_link_libraries(my_python_module ${Python_LIBRARIES})
endif()

This works fine in Linux, but in Windows, it only works in Release mode, not in Debug, in which case I always get a:

LINK : fatal error LNK1104: Can't open file 'python37.lib'

After some hair pulling I noticed the issue was caused by CMake instructing Visual Studio to link against 'python37_d.lib' instead of 'python37.lib' in the Debug mode.

However, as I described in the linked issue, the officially provided Boost Python debug library is linked against the Python release library, not the debug one. So, the solution would be to force the link against the Python release library, regardless of the build type. Unfortunately, ${Python_LIBRARIES} sets the library automatically depending on the mode, and I wouldn't like to explicitly specify python37.lib in my code (since I can upgrade Python and I don't want to have to change my CMake scripts because of that).

I found some similar issues here and here, but that doesn't reflect the exact situation I'm facing. Based on those, I tried setting:

target_link_libraries(my_python_module optimized ${Python_LIBRARIES})

But that didn't work either. So, the question is:

Is there a way to force the usage of the Python release library in Debug mode WITHOUT having to set it explicitly and leaving the Python CMake package to do it automatically instead. By explicit I mean doing:

target_link_libraries(my_python_module python37)

Thanks a lot for your help.

Puebla

I don't think there's a way to do this. You could build a debug version of Python to use with a debug version of the module. The problem is that the interpreter and any code linked into it all need to be using with the same C/C++ runtime library (there are both debug and release of of it). – Modicum

According to FindBoost.cmake here, you can set a variable Boost_USE_DEBUG_PYTHON before calling find_package(Boost ...). Try setting that variable to OFF set(Boost_USE_DEBUG_PYTHON OFF). Does that help? – Basin

@martineau, I think you misunderstood my question. I already have the Boost precompiled libraries, and the Boost.Python debug one was indeed compiled linking against the Python release library (python37.lib), not the debug one (python37_d.lib). But, since FindPython.cmake automatically sets the proper Python library for each build type, my code is trying to link against python37_d.lib in debug mode, when I have to do it against python37.lib. In fact, forcing the linkage against python37.lib in debug fixes the issue, but that implies changing my CMake config and I'd prefer not. – Puebla

@kanstar, unfortunately not, Boost_USE_DEBUG_PYTHON didn't fix the issue. – Puebla

@A.Palma Sorry, I misunderstood your question. With my suggestion, you will only get the release version of libboost_python and not libpython. Did you have a look at FindPython.cmake? The variable Python_FIND_ABI looks interesting (see here). Try set(Python_FIND_ABI "OFF" "ANY" "ANY") (will set debug to OFF and malloc and unicode to ANY) before doing find_package(Python). – Basin

Of course, that would be too easy: Python ships with pyconfig set to use pragmas to inject a pythonx_d.lib dependency: github.com/python/cpython/blob/3.10/PC/pyconfig.h#L270-L276 – Deane

Answer 1

It seems that set(Python_FIND_ABI "OFF" "ANY" "ANY") as suggested in the comments by kanstar would be the correct way to do this. However, while Python_FIND_ABI is in CMake master, it hasn't been released yet in the latest version (v3.15.2 as of this writing).

In the meantime, there are solutions dependent on the CMake version.

CMake 3.12 and above

It's possible to link against FindPython's Python_LIBRARY_RELEASE, which isn't meant to be part of the module's public interface, but the variable is set correctly nonetheless.

cmake_minimum_required (VERSION 3.12)
find_package(Python ..<choose your COMPONENTS; refer to FindPython docs>..)
if(WIN32 AND MSVC)
  target_link_libraries(my_python_module ${Python_LIBRARY_RELEASE})
endif()

CMake 3.0.4 to 3.11

Thanks to a comment by @Phil, we can expand the answer to include earlier CMake versions which had the FindPythonLibs module that sets the PYTHON_LIBRARY_RELEASE variable instead.

cmake_minimum_required (VERSION 3.0)
find_package(PythonLibs ..<refer to FindPythonLibs docs>..)
if(WIN32 AND MSVC)
  target_link_libraries(my_python_module ${PYTHON_LIBRARY_RELEASE})
endif()

Donetsk

I believe that should be ${PYTHON_LIBRARY_RELEASE} in all caps. – Empoison

@Empoison Is there a case where Python_LIBRARY_RELEASE doesn't work? Python_*, Python2_*, and Python3_* are the prefixes to use according to FindPython docs. – Donetsk

@Empoison Ah, I think you might be referring to FindPythonLibs, which does have the token PYTHON_LIBRARY_RELEASE (source), but the entire module's deprecated since 3.12. – Donetsk

Yes, I think you are right. This is what the pybind11 example uses, and I am blindly following that. Thanks for your help and the excellent answer! – Empoison

what about visual studio? – Disaffection

@Disaffection assuming for example Python 3.7; in Visual Studio go to <your project> -> Properties -> (top left) set Configuration: Debug. Then in Linker -> Input, make sure the Additional Dependencies field has python37.lib and not python37_d.lib – Donetsk

@Konafa, yes, but that won't be reflected in CMakeList, but would only work if used with visual studio solution file – Disaffection

The original answer already covers cmake. I probably misunderstood your question then; help me understand what you mean. – Donetsk

Windows 10, VS2019, Python 3.10: this doesn't work, with Py310 installed into C:\Python310: ``` LIST(INSERT CMAKE_PREFIX_PATH 0 "C:\\Python310") SET(Python_USE_SHARED_LIBS OFF) SET(Python_IMPLEMENTATIONS CPython) SET(Python_FIND_ABI "OFF" "ANY" "ANY") FIND_PACKAGE(Python3 3.10.0 REQUIRED COMPONENTS Development.embed) ``` fails to find python in debug build, ``` – Deane