Force CMake/VisualStudio to use static libs of Boost.Python

Question

I'm currently trying to build on Windows (with Intel compiler) a big project compiling very well on UNIX with CMake. Here is a reduced simple example of my problem.

Running the following simple example of code using Boost.Python :

#include <iostream>
#include <Python.h>
#include <boost/python.hpp>

int main()
{
    std::string python_home = "C:\\softs\\python\\2.7.9\\64";
    char* python_home_char = new char[python_home.length() + 1];
    strcpy(python_home_char, python_home.c_str());
    Py_SetPythonHome(python_home_char);
    Py_Initialize();

    boost::python::object pyobj_main = boost::python::import("__main__");
    boost::python::object glob = pyobj_main.attr("__dict__");
    boost::python::exec("print \"Hello from Python\"", glob, glob);

    Py_Finalize();

    return 0;
}

I get the following error message when executing the code :

The program can't start because boost_python-iw-mt-1_57.dll is missing from your computer. Try reinstalling the program to fix this problem.

Note that I don't have the error if a remove the 3 lines boost::python::*** at the end of the code. Also, I don't have any error if I compile the following example using Boost.Thread (In my mind, it also use libraries (not header only), is it right ?) :

#include <iostream>
#include <boost/thread.hpp>

boost::mutex mutex_hello;
void hello(unsigned long int thread_number)
{
    boost::mutex::scoped_lock lock_hello(mutex_hello);
    std::cout << "Hello from thread " << thread_number << std::endl;
}

int main()
{
    boost::thread_group group;
    for(unsigned long int i = 0; i < 9; ++i)
        group.create_thread(boost::bind(hello, i + 1));
    group.join_all();

    return 0;
}

Actually, I don't want to use shared libraries, I want my executable more static as possible.

I build this code with the following CMake file :

cmake_minimum_required(VERSION 2.8.9)

# Project
project(TestBoost)
enable_language(C)
enable_language(CXX)
enable_language(Fortran)

# Compiler info
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
    set(CompilerName "${CMAKE_CXX_COMPILER_ID}")
    set(CompilerVersion "${CMAKE_CXX_COMPILER_VERSION}")
    if(CMAKE_CL_64)
        set(CompilerArch "64")
    else()
        set(CompilerArch "32")
    endif()
endif()
string(TOLOWER "${CompilerName}" CompilerName)
if("${CompilerVersion}" MATCHES "([0-9]+\\.[0-9]+\\.[0-9]+)\\..*")
    string(REGEX REPLACE "([0-9]+\\.[0-9]+\\.[0-9]+)\\..*" "\\1" CompilerVersion "${CompilerVersion}")
endif()

# Libs
set(CMAKE_FIND_LIBRARY_SUFFIXES .lib .a ${CMAKE_FIND_LIBRARY_SUFFIXES})

# Intel
if(CompilerName STREQUAL "intel")
    string(REGEX REPLACE "(.*)/bin/.*" "\\1" IntelPath "${CMAKE_C_COMPILER}")
    if(CompilerArch STREQUAL "32")
        set(IntelArchStr "ia32")
    elseif(CompilerArch STREQUAL "64")
        set(IntelArchStr "intel64")
    endif()
    set(Compiler_LIBRARY_DIRS "${IntelPath}/compiler/lib/${IntelArchStr}")
endif()

# Boost
set(BOOST_ROOT "C:/softs/boost/1.57.0/${CompilerArch}/${CompilerName}/${CompilerVersion}")
set(Boost_USE_MULTITHREAD ON)
set(Boost_USE_STATIC_LIBS ON)
set(Boost_NO_SYSTEM_PATHS ON)
set(Boost_ADDITIONAL_VERSIONS "1.57.0" "1.57")
find_package(Boost 1.57.0 REQUIRED COMPONENTS thread system filesystem python)

# Python
set(PythonPath C:/softs/python/2.7.9/${CompilerArch})
set(Python_INCLUDE_DIRS ${PythonPath}/include)
set(Python_LIBRARY_DIRS ${PythonPath}/libs)
set(Python_LIBRARIES python27)

# Executable
include_directories(SYSTEM ${Boost_INCLUDE_DIRS} ${Python_INCLUDE_DIRS})
link_directories(${Boost_LIBRARY_DIRS} ${Python_LIBRARY_DIRS} ${Compiler_LIBRARY_DIRS})
add_executable(test_boost test_boost.cpp)
target_link_libraries(test_boost ${Boost_LIBRARIES} ${Python_LIBRARIES})

I run this CMake script with an initial entry CMAKE_GENERATOR_TOOLSET set to Intel C++ Compiler XE 14.0, and specifying Intel compiler as native compiler. I have no error or warning while running CMake on it (see the log below). This generates a file TestBoost.sln. When loaded in Visual Studio Professional 2013, I change the compilation type to Release and modify the linker options of the target test_boost to add /NODEFAULTLIB:LIBCMT (workaround due to a bug in CMake). Then I compile without any error.

Log from CMake :

The C compiler identification is Intel 14.0.4.20140805
The CXX compiler identification is Intel 14.0.4.20140805
Check for working C compiler using: Visual Studio 12 2013 Win64
Check for working C compiler using: Visual Studio 12 2013 Win64 -- works
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Check for working CXX compiler using: Visual Studio 12 2013 Win64
Check for working CXX compiler using: Visual Studio 12 2013 Win64 -- works
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
The Fortran compiler identification is Intel
Check for working Fortran compiler using: Visual Studio 12 2013 Win64
Check for working Fortran compiler using: Visual Studio 12 2013 Win64  -- works
Detecting Fortran compiler ABI info
Detecting Fortran compiler ABI info - done
Determine Intel Fortran Compiler Implicit Link Path
Determine Intel Fortran Compiler Implicit Link Path -- done
Checking whether C:/Program Files (x86)/Intel/Composer XE 2013 SP1/bin/intel64/icl.exe supports Fortran 90
Checking whether C:/Program Files (x86)/Intel/Composer XE 2013 SP1/bin/intel64/icl.exe supports Fortran 90 -- yes
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:515 ] _boost_TEST_VERSIONS = 1.57.0;1.57
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:517 ] Boost_USE_MULTITHREADED = TRUE
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:519 ] Boost_USE_STATIC_LIBS = ON
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:521 ] Boost_USE_STATIC_RUNTIME = 
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:523 ] Boost_ADDITIONAL_VERSIONS = 1.57.0;1.57
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:525 ] Boost_NO_SYSTEM_PATHS = ON
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:577 ] Declared as CMake or Environmental Variables:
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:579 ]   BOOST_ROOT = C:/softs/boost/1.57.0/64/intel/14.0.4
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:581 ]   BOOST_INCLUDEDIR = 
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:583 ]   BOOST_LIBRARYDIR = 
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:585 ] _boost_TEST_VERSIONS = 1.57.0;1.57
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:654 ] Include debugging info:
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:656 ]   _boost_INCLUDE_SEARCH_DIRS = C:/softs/boost/1.57.0/64/intel/14.0.4/include;C:/softs/boost/1.57.0/64/intel/14.0.4;NO_CMAKE_SYSTEM_PATH
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:658 ]   _boost_PATH_SUFFIXES = boost-1_57_0;boost_1_57_0;boost/boost-1_57_0;boost/boost_1_57_0;boost-1_57;boost_1_57;boost/boost-1_57;boost/boost_1_57
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:678 ] location of version.hpp: C:/softs/boost/1.57.0/64/intel/14.0.4/include/boost-1_57/boost/version.hpp
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:702 ] version.hpp reveals boost 1.57.0
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:787 ] guessed _boost_COMPILER = -iw
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:797 ] _boost_MULTITHREADED = -mt
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:840 ] _boost_RELEASE_ABI_TAG = -
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:842 ] _boost_DEBUG_ABI_TAG = -gd
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:890 ] _boost_LIBRARY_SEARCH_DIRS = C:/softs/boost/1.57.0/64/intel/14.0.4/lib;C:/softs/boost/1.57.0/64/intel/14.0.4/stage/lib;C:/softs/boost/1.57.0/64/intel/14.0.4/include/boost-1_57/lib;C:/softs/boost/1.57.0/64/intel/14.0.4/include/boost-1_57/../lib;C:/softs/boost/1.57.0/64/intel/14.0.4/include/boost-1_57/stage/lib;NO_CMAKE_SYSTEM_PATH
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1001 ] Searching for THREAD_LIBRARY_RELEASE: libboost_thread-iw-mt-1_57;libboost_thread-iw-mt;libboost_thread-mt-1_57;libboost_thread-mt;libboost_thread;libboost_thread-iw-mt-s-1_57;libboost_thread-iw-mt-s;libboost_thread-mt-s-1_57;libboost_thread-mt-s
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1037 ] Searching for THREAD_LIBRARY_DEBUG: libboost_thread-iw-mt-gd-1_57;libboost_thread-iw-mt-gd;libboost_thread-mt-gd-1_57;libboost_thread-mt-gd;libboost_thread-mt;libboost_thread;libboost_thread-iw-mt-s-gd-1_57;libboost_thread-iw-mt-s-gd;libboost_thread-mt-s-gd-1_57;libboost_thread-mt-s-gd
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1001 ] Searching for SYSTEM_LIBRARY_RELEASE: libboost_system-iw-mt-1_57;libboost_system-iw-mt;libboost_system-mt-1_57;libboost_system-mt;libboost_system;libboost_system-iw-mt-s-1_57;libboost_system-iw-mt-s;libboost_system-mt-s-1_57;libboost_system-mt-s
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1037 ] Searching for SYSTEM_LIBRARY_DEBUG: libboost_system-iw-mt-gd-1_57;libboost_system-iw-mt-gd;libboost_system-mt-gd-1_57;libboost_system-mt-gd;libboost_system-mt;libboost_system;libboost_system-iw-mt-s-gd-1_57;libboost_system-iw-mt-s-gd;libboost_system-mt-s-gd-1_57;libboost_system-mt-s-gd
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1001 ] Searching for FILESYSTEM_LIBRARY_RELEASE: libboost_filesystem-iw-mt-1_57;libboost_filesystem-iw-mt;libboost_filesystem-mt-1_57;libboost_filesystem-mt;libboost_filesystem;libboost_filesystem-iw-mt-s-1_57;libboost_filesystem-iw-mt-s;libboost_filesystem-mt-s-1_57;libboost_filesystem-mt-s
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1037 ] Searching for FILESYSTEM_LIBRARY_DEBUG: libboost_filesystem-iw-mt-gd-1_57;libboost_filesystem-iw-mt-gd;libboost_filesystem-mt-gd-1_57;libboost_filesystem-mt-gd;libboost_filesystem-mt;libboost_filesystem;libboost_filesystem-iw-mt-s-gd-1_57;libboost_filesystem-iw-mt-s-gd;libboost_filesystem-mt-s-gd-1_57;libboost_filesystem-mt-s-gd
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1001 ] Searching for PYTHON_LIBRARY_RELEASE: libboost_python-iw-mt-1_57;libboost_python-iw-mt;libboost_python-mt-1_57;libboost_python-mt;libboost_python;libboost_python-iw-mt-s-1_57;libboost_python-iw-mt-s;libboost_python-mt-s-1_57;libboost_python-mt-s
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1037 ] Searching for PYTHON_LIBRARY_DEBUG: libboost_python-iw-mt-gd-1_57;libboost_python-iw-mt-gd;libboost_python-mt-gd-1_57;libboost_python-mt-gd;libboost_python-mt;libboost_python;libboost_python-iw-mt-s-gd-1_57;libboost_python-iw-mt-s-gd;libboost_python-mt-s-gd-1_57;libboost_python-mt-s-gd
[ C:/Program Files (x86)/CMake/share/cmake-3.1/Modules/FindBoost.cmake:1088 ] Boost_FOUND = 1
Boost version: 1.57.0
Found the following Boost libraries:
  thread
  system
  filesystem
  python
Configuring done

How can I force the compilation to use only static libraries ? I thought it was the case with the options in the CMake file. Moreover, it's working with the example using threads... Maybe something is wrong in the installation of Boost.Python ?

Aurelea

In visual studio, if you look at C++/Code Generation, is it set to multi-threaded or multithreaded DLL? The problem is if you change this, you may end up with something incompatible with the python libraries unless you have the source of the python libraries and can change the code generation of that too. – Cowitch

First, obvious question: Do you actually have a static version of boost at C:/softs/boost/1.57.0/? You can use Boost_DEBUG=ON to "debug" during CMake what find_package( Boost ) finds.... – Drayage

@Andre : Yes I have. I added the log from CMake in my question. But, maybe are they not well compiled ? How to know that ? – Aurelea

@Cowitch : Yes, it's set to "Multi-threaded DLL (/MD)". If I try to change it to "Multi-threaded (/MT)", I get the following error : "Mixing a dll boost library with a static runtime is a really bad idea...". Then a solution could be rebuild python myself ? – Aurelea

You use the words "static" and "shared" libraries quite open minded :) That said, if you have both static and shared versions of boost and still can't link, then maybe giving a full path to the lib-file might help at target_link_libraries. Regarding the \MT you would have to make the same change to all dependencies that link CRT. – Goldengoldenberg

@Goldengoldenberg : Ho! You're right. I changed my title. What is "CRT" ? – Aurelea

CRT = C run time libraries in Visual Studio. e.g., msvcr100.dll & msvcp100.dll. – Goldengoldenberg

Or in this case with VS2013, msvcr120.dll etc. Note if you switch to VS2015, it is msvcr140.dll – Cowitch

I do not have an environment in which I can test, but does defining BOOST_PYTHON_STATIC_LIB resolve the problem (either via add_definitions() in CMake or #define before including boost.python.hpp)? Boost.Python is an exception to the Boost_USE_STATIC_LIBS variable, and its default config is to auto-link against the dynamic library. – Kindred

@TannerSansbury : It works ! Thanks a lot ! Please, add your solution as a response to have the bounty. – Aurelea

Answer 1

When using Boost.Python, the default configuration is dynamic linking. To force static linking, define BOOST_PYTHON_STATIC_LIB in the compilation. Consider either:

• Add add_definitions(-DBOOST_PYTHON_STATIC_LIB) within CMake
• Add #define BOOST_PYTHON_STATIC_LIB before including boost/python.hpp
• Add #define BOOST_PYTHON_STATIC_LIB in boost/config/user.hpp

---

Boost.Python is an exception to the Boost_USE_STATIC_LIBS CMake variable. The FindBoost documentation notes:

On Visual Studio and Borland compilers Boost headers request automatic linking to corresponding libraries. [...] Boost automatic linking typically requests static libraries with a few exceptions (such as Boost.Python).

Unfortunately, the BOOST_PYTHON_STATIC_LIB option is neither listed in the Boost user settable options, Choosing a Boost.Python Library Binary, nor Boost.Python Configuration Information documentation.

---

A few other recommendations given your example code:

• Per Boost.Python's embedding documentation, do not invoke Py_Finalize(). Some internal Boost.Python objects will remain alive during Py_Finalize(), and only attempt to be deleted when Boost.Python unloads, causing the objects to attempt deletion with a non-existent interpreter.
• Do not directly include, python.h. If you must, consider including boost/python/detail/wrap_python.hpp instead.
• Do not include any system headers before python.h (or the Boost.Python files). This restriction is imposed by Python (see here).

The latter two recommendations documented in the #include issues section.

Kindred

Answer 2

It seems like you have a problem with finding dll's on your machine. As a temporary fix, you can try to rename your .dll file to .pyd.

Peirce

As I said, I don't want tu use shared libs, but only static. The title of my question was wrong. I corrected it. – Aurelea