Getting CMake to build out of source without wrapping scripts

Question

I'm trying to get CMake to build into a directory 'build', as in project/build, where the CMakeLists.txt is in project/.

I know I can do:

mkdir build
cd build
cmake ../

but that is cumbersome. I could put it in a script and call it, but then it's unpleasant to provide different arguments to CMake (like -G "MSYS Makefiles"), or I would need to edit this file on each platform.

Preferably I would do something like SET(CMAKE_OUTPUT_DIR build) in the main CMakeLists.txt. Please tell me that this is possible, and if so, how? Or some other out of source build method that makes it easy to specify different arguments?

Thigmotropism

I have the same project architecture as you. My "build" dir is always here. I personnaly find that typing cmake .. is not such a big deal. – Damarisdamarra

Answer 1

CMake 3.13 or newer supports the command line options -S and -B to specify source and binary directory, respectively.

cmake -S . -B build -G "MSYS Makefiles"

This will look for the CMakeLists.txt in the current folder and create a build folder (if it does not yet exist) in it.

For older versions of CMake, you can use the undocumented CMake options -H and -B to specify the source and binary directory upon invoking cmake:

cmake -H. -Bbuild -G "MSYS Makefiles"

Note that there must not be a space character between the option and the directory path.

Nucleate

Nice options. But Max said "it's unpleasant to provide different arguments to cmake". He wants it into the CMakeList. – Damarisdamarra

@Damarisdamarra Actually, Max said the opposite. He said it's unpleasant to provide arguments via a file because then the file has to be edited on every platform. But that assumes the file is checked into Git and shared with other developers. If it's not shared (or only the common parts are shared like in my answer) then it's not a problem. – Cashbook

Answer 2

A solution that I found recently is to combine the out-of-source build concept with a Makefile wrapper.

In my top-level CMakeLists.txt file, I include the following to prevent in-source builds:

if ( ${CMAKE_SOURCE_DIR} STREQUAL ${CMAKE_BINARY_DIR} )
    message( FATAL_ERROR "In-source builds not allowed. Please make a new directory (called a build directory) and run CMake from there. You may need to remove CMakeCache.txt." )
endif()

Then, I create a top-level Makefile, and include the following:

# -----------------------------------------------------------------------------
# CMake project wrapper Makefile ----------------------------------------------
# -----------------------------------------------------------------------------

SHELL := /bin/bash
RM    := rm -rf
MKDIR := mkdir -p

all: ./build/Makefile
    @ $(MAKE) -C build

./build/Makefile:
    @  ($(MKDIR) build > /dev/null)
    @  (cd build > /dev/null 2>&1 && cmake ..)

distclean:
    @  ($(MKDIR) build > /dev/null)
    @  (cd build > /dev/null 2>&1 && cmake .. > /dev/null 2>&1)
    @- $(MAKE) --silent -C build clean || true
    @- $(RM) ./build/Makefile
    @- $(RM) ./build/src
    @- $(RM) ./build/test
    @- $(RM) ./build/CMake*
    @- $(RM) ./build/cmake.*
    @- $(RM) ./build/*.cmake
    @- $(RM) ./build/*.txt

ifeq ($(findstring distclean,$(MAKECMDGOALS)),)
    $(MAKECMDGOALS): ./build/Makefile
    @ $(MAKE) -C build $(MAKECMDGOALS)
endif

The default target all is called by typing make, and invokes the target ./build/Makefile.

The first thing the target ./build/Makefile does is to create the build directory using $(MKDIR), which is a variable for mkdir -p. The directory build is where we will perform our out-of-source build. We provide the argument -p to ensure that mkdir does not scream at us for trying to create a directory that may already exist.

The second thing the target ./build/Makefile does is to change directories to the build directory and invoke cmake.

Back to the all target, we invoke $(MAKE) -C build, where $(MAKE) is a Makefile variable automatically generated for make. make -C changes the directory before doing anything. Therefore, using $(MAKE) -C build is equivalent to doing cd build; make.

To summarize, calling this Makefile wrapper with make all or make is equivalent to doing:

mkdir build
cd build
cmake ..
make 

The target distclean invokes cmake .., then make -C build clean, and finally, removes all contents from the build directory. I believe this is exactly what you requested in your question.

The last piece of the Makefile evaluates if the user-provided target is or is not distclean. If not, it will change directories to build before invoking it. This is very powerful because the user can type, for example, make clean, and the Makefile will transform that into an equivalent of cd build; make clean.

In conclusion, this Makefile wrapper, in combination with a mandatory out-of-source build CMake configuration, make it so that the user never has to interact with the command cmake. This solution also provides an elegant method to remove all CMake output files from the build directory.

P.S. In the Makefile, we use the prefix @ to suppress the output from a shell command, and the prefix @- to ignore errors from a shell command. When using rm as part of the distclean target, the command will return an error if the files do not exist (they may have been deleted already using the command line with rm -rf build, or they were never generated in the first place). This return error will force our Makefile to exit. We use the prefix @- to prevent that. It is acceptable if a file was removed already; we want our Makefile to keep going and remove the rest.

Another thing to note: This Makefile may not work if you use a variable number of CMake variables to build your project, for example, cmake .. -DSOMEBUILDSUSETHIS:STRING="foo" -DSOMEOTHERBUILDSUSETHISTOO:STRING="bar". This Makefile assumes you invoke CMake in a consistent way, either by typing cmake .. or by providing cmake a consistent number of arguments (that you can include in your Makefile).

Finally, credit where credit is due. This Makefile wrapper was adapted from the Makefile provided by the C++ Application Project Template.

This answer was originally posted here. I thought it applied to your situation as well.

Flit

I think this is generally a good solution. But it is using a wrapper script. While the OP was asking for a solution "without wrapping scripts". It is also won't work on windows. – Burned

This might be just me, but there's something I don't like about wrapping a build system generator with a specific build system like Make. – Rondi

A cross-platform solution is to write the wrapper script in CMake. See my answer. – Cashbook

Answer 3

Based on the previous answers, I wrote the following module that you can include to enforce an out-of-source build.

set(DEFAULT_OUT_OF_SOURCE_FOLDER "cmake_output")

if (${CMAKE_SOURCE_DIR} STREQUAL ${CMAKE_BINARY_DIR})
    message(WARNING "In-source builds not allowed. CMake will now be run with arguments:
        cmake -H. -B${DEFAULT_OUT_OF_SOURCE_FOLDER}
")

    # Run CMake with out of source flag
    execute_process(
            COMMAND ${CMAKE_COMMAND} -H. -B${DEFAULT_OUT_OF_SOURCE_FOLDER}
            WORKING_DIRECTORY ${CMAKE_SOURCE_DIR})

    # Cause fatal error to stop the script from further execution
    message(FATAL_ERROR "CMake has been ran to create an out of source build.
This error prevents CMake from running an in-source build.")
endif ()

This works, however I already noticed two downsides:

• When the user is lazy and simply runs cmake ., they will always see a FATAL_ERROR. I could not find another way to prevent CMake from doing any other operations and exit early.
• Any command line arguments passed to the original call to cmake will not be passed to the "out-of-source build call".

Suggestions to improve this module are welcome.

Cattleya

One way to avoid the fatal error is using an else() instead of your endif() and put the endif() at the very end of the original cmake code. – Burned

@Burned That is true but given that I put this in a separate, reusable module that is included by a CMakeLists.txt file, your suggestion is not applicable. – Cattleya

Answer 4

The trick is to write your wrapper script... in CMake!

A Bash script or Makefile wrapper would require developers to have Bash or Make installed on their system, but a script written in CMake will run on all platforms without introducing any new dependencies.

We can take advantage of CMake's -P option, which will:

Process the given cmake file as a script written in the CMake language. No configure or generate step is performed and the cache is not modified.

Inside the script, we'll use the file() command to create the build directory, then use execute_process() to run CMake again, this time with the options necessary to generate the build system. Since the options are specified in the script, this saves you having to type them manually on the command line each time.

---

Create a new text file called build.cmake in the project folder and paste in the following code:

#!/usr/bin/env -S cmake -P

# Set default values for project variables
set(SOURCE_PATH "${CMAKE_CURRENT_LIST_DIR}") # directory of this script
set(BUILD_PATH "${SOURCE_PATH}/build")

set(CPUS 16) # number of logical processor cores (for parallel builds)

# Set generator for each platform
if(WIN32)
    set(GENERATOR "MSYS Makefiles")
elseif(APPLE)
    set(GENERATOR "Xcode")
else()
    set(GENERATOR "Ninja")
endif()

# Allow developers to override the above variables if they want
if(EXISTS "${SOURCE_PATH}/overrides.cmake")
    include("${SOURCE_PATH}/overrides.cmake")
endif()

# Configure
if(NOT EXISTS "${BUILD_PATH}/CMakeCache.txt")
    file(MAKE_DIRECTORY "${BUILD_PATH}")
    execute_process(
        # CMAKE_ARGS on next line must not be quoted
        COMMAND "${CMAKE_COMMAND}" -S "${SOURCE_PATH}" -B . -G "${GENERATOR}" ${CMAKE_ARGS}
        WORKING_DIRECTORY "${BUILD_PATH}"
        RESULT_VARIABLE EXIT_STATUS
    )
    if(NOT "${EXIT_STATUS}" EQUAL "0")
        file(REMOVE "${BUILD_PATH}/CMakeCache.txt") # force CMake to run again next time
        message(FATAL_ERROR "CMake failed with status ${EXIT_STATUS}.")
    endif()
endif()

# Build
execute_process(
    COMMAND "${CMAKE_COMMAND}" --build . --parallel "${CPUS}"
    WORKING_DIRECTORY "${BUILD_PATH}"
    RESULT_VARIABLE EXIT_STATUS
)
if(NOT "${EXIT_STATUS}" EQUAL "0")
    message(FATAL_ERROR "${GENERATOR} failed with status ${EXIT_STATUS}.")
endif()

Important: The file build.cmake must use Unix-style LF (\n) line terminators for the shebang line to work. The -S option for env is required on some systems to allow multiple arguments to be specified on the shebang line.

Now you can build the project simply by running this on the command line:

# From any Unix shell (Linux, macOS, WSL, Git Bash)
./build.cmake

# CMD or PowerShell
cmake -P build.cmake

That's great, but the project variables were all hard-coded into build.cmake so they will be the same for all developers. What if I want to build the project with a different set of variables?

No problem! Just create another file called overrides.cmake in the same directory as build.cmake. Specify the values you need in overrides.cmake and they will take precendence over the ones in build.cmake.

For example, in overrides.cmake you might have:

set(CPUS 32) # override default CPUS set in build.cmake
set(GENERATOR "Visual Studio 17 2022") # use different generator

# Pass some options to CMake
set(CMAKE_ARGS
    "-DFIRST_OPTION=Value 1"
    "-DSECOND_OPTION=Value 2"
)

set(ENV{PATH} "C:/Tools;$ENV{PATH}") # add a directory to PATH

Important: The file overrides.cmake must be added to your project's .gitignore to allow each developer to specify their own overrides.

Take a look at MuseScore's build.cmake and developer documentation for more ideas, such as how to read and process arguments passed to the script on the command line, and how to use MSVC as the compiler.

Cashbook