From CMake setup 'make' to use '-j' option by default

Question

I want my CMake project to be built by make -j N, whenever I call make from the terminal. I don't want to set -j option manually every time.

For that, I set CMAKE_MAKE_PROGRAM variable to the specific command line. I use the ProcessorCount() function, which gives the number of procesors to perform build in parallel.

When I do make, I do not see any speed up. However if I do make -j N, then it is built definitely faster.

Would you please help me on this issue? (I am developing this on Linux.)

Here is the snippet of the code that I use in CMakeList.txt:

include(ProcessorCount)
ProcessorCount(N)
message("number of processors: "  ${N})
if(NOT N EQUAL 0)
  set(CTEST_BUILD_FLAGS -j${N})
  set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
  set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j ${N}")      
endif()
message("cmake make program" ${CMAKE_MAKE_PROGRAM})

Thank you very much.

Incalescent

None part of your code run anything. So, what do you want to speed up? Note, that cmake phase cannot be parallelisized: CMake scripts are processes sequentially, line by line, like in other scripting languages like python. As opposite, build phase may be parallelized and make -j does exactly this. – Biped

Some advice regarding CMake / build perfomance: https://mcmap.net/q/494018/-how-to-speed-up-compile-time-of-my-cmake-enabled-c-project/2799037 – Neese

I know it does not run, It is a segment of CMakeList.txt, I am simply trying to set the CMAKE_MAKE_PROGRAM and want to envoke "make -j N" each time that I call make from the terminal, instead of setting it explicitly in the terminal each time , – Incalescent

I would recommend to use ninja as a replacement for make. It's faster and uses all available cores for compiling/linking by default. – Depression

@DrJ: I have added your comment into the question post, as it clarifies the problem a lot. In the future, tend to do that by your hands. On Stack Overflow all information which describes the problem should be in the question post. Comments are temporary. – Biped

Trying to do this is not portable. – Litigation

How to use N-1 cores ? – Cahilly

Answer 1

Via setting the CMAKE_MAKE_PROGRAM variable you want to affect the build process. But:

1. This variable affects only the build via cmake --build, not on native tool (make) call:

   The CMAKE_MAKE_PROGRAM variable is set for use by project code. The value is also used by the cmake(1) --build and ctest(1) --build-and-test tools to launch the native build process.

2. This variable should be a CACHEd one. It is used in such way by make-like generators:

   These generators store CMAKE_MAKE_PROGRAM in the CMake cache so that it may be edited by the user.

   That is, you need to set this variable with

   set(CMAKE_MAKE_PROGRAM <program> CACHE PATH "Path to build tool" FORCE)
   

3. This variable should refer to the executable itself, not to a program with arguments:

   The value may be the full path to an executable or just the tool name if it is expected to be in the PATH.

   That is, value "make -j 2" cannot be used for that variable (splitting arguments as list

   set(CMAKE_MAKE_PROGRAM make -j 2 CACHE PATH "Path to build tool" FORCE)
   

   wouldn't help either).

In summary, you may redefine the behavior of cmake --build calls with setting the CMAKE_MAKE_PROGRAM variable to the script, which calls make with parallel options. But you may not affect the behavior of direct make calls.

Biped

Thank you for the detailed answer – Incalescent

You may create an alias for "make -jN" – Trahern

Answer 2

In case you want to speed up the build you can run multiple make processes in parallel but not cmake. To perform every build with predefined number of parallel processes you can define this in MAKEFLAGS.

Set MAKEFLAGS in your environment script, e.g. ~/.bashrc as you want:

export MAKEFLAGS=-j8

On Linux the following sets MAKEFLAGS to the number of CPUs - 1: (Keep one CPU free for other tasks while build) and is useful in environments with dynamic ressources, e.g. VMware:

export MAKEFLAGS=-j$(($(grep -c "^processor" /proc/cpuinfo) - 1))

New from cmake v3.12 on:

The command line has a new option --parallel <JOBS>.

Example:

cmake --build build_arm --parallel 4 --target all

Example with number of CPUs- 1 using nproc:

cmake --build build_arm --parallel $(($(nproc) - 1)) --target all

Abduction

Be warned that some shells (zsh) in extended glob mode expand ^processor as a glob pattern, make sure to use grep -c "^processor" /proc/cpuinfo. – Kostman

just using nproc instead of grep -c "^processor" /proc/cpuinfo may be more readable. – Politic

Answer 3

You may set the env variable MAKEFLAGS using this command

export MAKEFLAGS=-j$(nproc)

Pinard

@sanmai Works fine for me with 4.1. – Paolo

Answer 4

My solution is to have a small script which will run make include all sorts of other features, not just the number of CPUs.

I call my script mk and I do a chmod 755 mk so I can run it with ./mk in the root of my project. I also have a few flags to be able to run various things with a simple command line. For example, while working on the code and I get many errors, I like to pipe the output to less. I can do that with ./mk -l without having to retype all the heavy duty Unix stuff...

As you can see, I have the -j4 in a couple of places where it makes sense. For the -l option, I don't want it because in this case it would eventually cause multiple errors to be printed at the same time (I tried that before!)

#!/bin/sh -e
#
# Execute make

case "$1" in
"-l")
    make -C ../BUILD/Debug 2>&1 | less -R
    ;;

"-r")
    make -j4 -C ../BUILD/Release
    ;;

"-d")
    rm -rf ../BUILD/Debug/doc/lpp-doc-?.*.tar.gz \
           ../BUILD/Debug/doc/lpp-doc-?.*
    make -C ../BUILD/Debug
    ;;

"-t")
    make -C ../BUILD/Debug
    ../BUILD/Debug/src/lpp tests/suite/syntax-print.logo
    g++ -std=c++14 -I rt l.cpp rt/*.cpp
    ;;

*)
    make -j4 -C ../BUILD/Debug
    ;;

esac

# From the https://github.com/m2osw/lpp project

With CMake, it wouldn't work unless, as Tsyvarev mentioned, you create your own script. But I personally don't think it's sensible to call make from your make script. Plus it could break a build process which would not expect that strange script. Finally, my script, as I mentioned, allows me to vary the options depending on the situation.

Vincenzovincible

Answer 5

I usually use alias in linux to set cm equal to cmake .. && make -j12. Or write a shell to specify make and clean progress ...

alias cm='cmake .. && make -j12'

Then use cm to make in a single command.

Remitter

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Lavin