CMake ExternalProject_Add() and FindPackage()

Question

Is there are proper way to find a library (via FindPackage()) which was built with ExternalProject_Add()?

The problem is that CMake cannot find the library at CMake-time because the external library gets build at compile time. I know that it is possible to combine these two CMake function when building the library and the project in a superbuild but I want to use it in a normal CMake project.

In fact I would like to build VTK 6 with ExternalProject_Add and find it with FindPackage all inside my CMake project.

Inoue

Answer 1

there is a way to do this. but it´s kind of hackish. you basically add a custom target, that reruns cmake during build.

you will have to try this in a small test project, to decide if it works for you

find_package(Beaengine)


############################################
#
#    BeaEngine
#
include(ExternalProject)
externalproject_add(BeaEngine
    SOURCE_DIR            ${PROJECT_SOURCE_DIR}/beaengine   
    SVN_REPOSITORY        http://beaengine.googlecode.com/svn/trunk/
    CMAKE_ARGS            -DoptHAS_OPTIMIZED=TRUE -DoptHAS_SYMBOLS=FALSE -DoptBUILD_64BIT=FALSE -DoptBUILD_DLL=FALSE -DoptBUILD_LITE=FALSE
    INSTALL_COMMAND       ""
 )


if(NOT ${Beaengine_FOUND})
    #rerun cmake in initial build
    #will update cmakecache/project files on first build
    #so you may have to reload project after first build
    add_custom_target(Rescan ${CMAKE_COMMAND} ${CMAKE_SOURCE_DIR} DEPENDS BeaEngine)
else()
    #Rescan becomes a dummy target after first build
    #this prevents cmake from rebuilding cache/projects on subsequent builds
    add_custom_target(Rescan)
endif()




add_executable(testapp testapp.cpp )
add_dependencies(testapp Rescan)
if(${Beaengine_FOUND})
    target_link_libraries(testapp ${Beaengine_LIBRARY})
endif()

this seems to work well for mingw makefiles / eclipse makefile projects. vs will request to reload all projects after first build.

Acalia

This mostly worked, but when using a FindXpackage that has cached variables, you might need to unset them before the find_package... ie, if building a newer version of bison than available on the system: unset (FLEX_EXECUTABLE CACHE) find_package (FLEX 2.6) unset (BISON_EXECUTABLE CACHE) find_package (BISON 3) – Zoan

Answer 2

You can force a build using the build_external_project function below.

It works by generating a simple helper project inside the build tree and then calling the cmake configuration and the cmake build on the helper.

Customize at will for the actual ExternalProject_add command.

Note that the trailing arguments are used to pass CMAKE_ARGS. Furthur enhancements are left as an exercise to the reader :-)

# This function is used to force a build on a dependant project at cmake configuration phase.
# 
function (build_external_project target prefix url) #FOLLOWING ARGUMENTS are the CMAKE_ARGS of ExternalProject_Add

    set(trigger_build_dir ${CMAKE_BINARY_DIR}/force_${target})

    #mktemp dir in build tree
    file(MAKE_DIRECTORY ${trigger_build_dir} ${trigger_build_dir}/build)

    #generate false dependency project
    set(CMAKE_LIST_CONTENT "
        cmake_minimum_required(VERSION 2.8)

        include(ExternalProject)
        ExternalProject_add(${target}
            PREFIX ${prefix}/${target}
            URL ${url}
            CMAKE_ARGS ${ARGN}
            INSTALL_COMMAND \"\"
            )

        add_custom_target(trigger_${target})
        add_dependencies(trigger_${target} ${target})
    ")

    file(WRITE ${trigger_build_dir}/CMakeLists.txt "${CMAKE_LIST_CONTENT}")

    execute_process(COMMAND ${CMAKE_COMMAND} ..
        WORKING_DIRECTORY ${trigger_build_dir}/build
        )
    execute_process(COMMAND ${CMAKE_COMMAND} --build .
        WORKING_DIRECTORY ${trigger_build_dir}/build
        )

endfunction()

Chromite

I have something similar in my project. It worth noting that if you want to build release/debug library for different generators you need different functions. For make-like generators you need two ExternalProject_Add commands, and for msvc-like generators you need one ExternalProject_Add command but with two install stages. – Incoordination

Don't forget to pass current generator to the external project with "execute_process(COMMAND ${CMAKE_COMMAND} -G${CMAKE_GENERATOR} .. " otherwise you may get a lot of headache if the binaries will be built with different generators :) – Bonnes

Answer 3

Time has passed and CMake implemented a native version allowing to reference targets from an ExternalProject_Add.

This feature is implemented in the FetchContent module. It allows downloading and immediately consuming targets defined at configure time.

It uses a scratch build dir as hinted by my previous answer, but in a more integrated API.

Chromite

How does this help with find_package? FetchContent doesn't provide a way to install the target, so how? – Invalid

The original question was about downloading a project to find it in the configure phase. Reading the doc on FetchContent allows us to see that this feature is also available in the FetchContent_MakeAvailable() function. – Chromite

To use find_package, it usually checks the actual file existence, for example, using find_library for lib and find_path for header. Without triggering the compile and install process, these files won't be exist at all. Also in the doc it said "INSTALL_COMMAND" is prohibited. Using FetchContent_MakeAvailable won't make the install happen. – Invalid

Maybe, but this question isn’t related to install, it is related to consuming a dynamically downloaded target at configure time. – Chromite

This is the perfect answer for code which is consumable directly at configure time (e.g. via add_subdirectory()). But some projects do not work in this way, especially VTK still needs to be consumed via ExternalProject_Add(). – Inoue