Automatically add all files in a folder to a target using CMake?

Question

I am considering switching a cross platform project from separate build management systems in Visual C++, XCode and makefiles to CMake.

One essential feature I need is to add automatically all files in a directory to a target. While this is easy to do with make, it is not easily doable with Visual C++ and XCode (correct me if I am wrong). Is it possible to do it in directly in CMake? How?

Colorist

Btw. in In Visual Studio, at least in C# projects, there's a toolbar button in the project explorer, named show all files. It makes all sub-directories of a project visible, greyed out if they don't contain any files logically included in the project. You can include these directories via the context menu, which includes all source files inside them, recursively :) – Eggleston

Possible duplicate of How to use all *.c files in a directory with the Cmake build system? – Interjection

Answer 1

As of CMake 3.1+ the developers strongly discourage users from using file(GLOB or file(GLOB_RECURSE to collect lists of source files.

Note: We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate. The CONFIGURE_DEPENDS flag may not work reliably on all generators, or if a new generator is added in the future that cannot support it, projects using it will be stuck. Even if CONFIGURE_DEPENDS works reliably, there is still a cost to perform the check on every rebuild.

See the documentation here.

There are two goods answers ([1], [2]) here on SO detailing the reasons to manually list source files.

---

It is possible. E.g. with file(GLOB:

cmake_minimum_required(VERSION 2.8)

file(GLOB helloworld_SRC
     "*.h"
     "*.cpp"
)

add_executable(helloworld ${helloworld_SRC})

Note that this requires manual re-running of cmake if a source file is added or removed, since the generated build system does not know when to ask CMake to regenerate, and doing it at every build would increase the build time.

As of CMake 3.12, you can pass the CONFIGURE_DEPENDS flag to file(GLOB to automatically check and reset the file lists any time the build is invoked. You would write:

cmake_minimum_required(VERSION 3.12)

file(GLOB helloworld_SRC CONFIGURE_DEPENDS "*.h" "*.cpp")

This at least lets you avoid manually re-running CMake every time a file is added.

Hoxie

An alternative to manually re-running cmake is to touch the CMakeLists.txt file before running make. – Whitcher

Alternative to GLOB: file(GLOB_RECURSE variable [RELATIVE path] [FOLLOW_SYMLINKS] [globbing expressions]...). – Sharma

>"We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate." This is no longer recommeneded in CMAKE3.7 docs linked about by Hand1Cloud – Ogg

@Ogg GLOB, not GLOB_RECURSE. – Sharma

whats the alternative to GLOB then? – Nootka

@Nootka you add the actual list of named files to cmake. It's a huge win in building bigger projects, and a trivial cost for smaller projects. – Curlpaper

And an absolute nightmare when trying to port an existing project with thousands of files. – Nootka

@Nootka you've definitely got it the wrong way around. If you have thousands of files, you really want to do things the CMake way, and have them all named in the cmake files correctly. It's a trivial shell script to build them. If you really, deeply don't want to do things the cmake way, you probably don't want to use cmake at all. It's not the only game in town. – Curlpaper

Its not the only game in town .... except when it is. (Ie using C-Lion or integrating into an existing cmake system – Nootka

A better way since v3.1 is suggested here: target_sources() crascit.com/2016/01/31/… – Flapjack

It doesn't work at all, tried multiple time and didn't work – Daradarach

"Note that this requires manual re-running of cmake if a source file is added or removed", but wouldn't you need to manually modify the CMakeLists.txt file in this case anyway?? And then still rerun cmake on the modified file? – Equalize

I laughed reading this answer. I've come across countless packages where it uses globs along with the obligatory comment "do as I say not as I do." – Frontage

I am shocked by how primitive modern tools are. CMake, for instance, could optionally just query Git and detect changes, or keep track of the source changes inside the build directory. I'll never understand how CMake became so popular other than reports that it sucks less than the alternatives. – Frontage

Answer 2

The answer by Kleist certainly works, but there is an important caveat:

When you write a Makefile manually, you might generate a SRCS variable using a function to select all .cpp and .h files. If a source file is later added, re-running make will include it.

However, CMake (with a command like file(GLOB ...)) will explicitly generate a file list and place it in the auto-generated Makefile. If you have a new source file, you will need to re-generate the Makefile by re-running cmake.

edit: No need to remove the Makefile.

Pride

Can't cmake simply be re-run, which should remove/modify the out-of-date Makefile? – Inveracity

This isn't an answer, it's a comment about @Kleist's answer. – Daggerboard

@Daggerboard not in Stackoverflow terminology. – Pride

This comment is nice, but it's "standard", right? The asker is using cmake already, so suggesting make can be "smarter" isn't that helpful. – Aposematic

So is there way to add new source files and then build without re-running cmake? – Solitude

I guess you can manually modify the Makefile auto-generated by CMake. But why? – Pride

create a bash /powershell script to output all .h .cpp files already in the format you need. Script coming soon – Ainsley

Answer 3

Extension for @Kleist answer:

Since CMake 3.12 additional option CONFIGURE_DEPENDS is supported by commands file(GLOB) and file(GLOB_RECURSE). With this option there is no needs to manually re-run CMake after addition/deletion of a source file in the directory - CMake will be re-run automatically on next building the project.

However, the option CONFIGURE_DEPENDS implies that corresponding directory will be re-checked every time building is requested, so build process would consume more time than without CONFIGURE_DEPENDS.

Even with CONFIGURE_DEPENDS option available CMake documentation still does not recommend using file(GLOB) or file(GLOB_RECURSE) for collect the sources.

Jurisprudence

"CMake documentation still does not recommend using file(GLOB) or file(GLOB_RECURSE) for collect the sources" What do they recommend then? Using something else to collect the sources or to manually list every single source file? – Monteverdi

@Thomas: They recommend to manually list every source file. That answer gives a great explanation of the recommendation. – Jurisprudence

Answer 4

To use Visual Studio project hierarchy inside Clion with cmake:

cmake_minimum_required(VERSION 3.17)
project(MyProject)

set(CMAKE_CXX_STANDARD 17)

file(GLOB APP_SOURCES */*.cpp)
foreach (testsourcefile ${APP_SOURCES})
    get_filename_component(testname ${testsourcefile} NAME_WLE)
    get_filename_component(dirname ${testsourcefile} DIRECTORY)
    file(GLOB dir_src CONFIGURE_DEPENDS
            "${dirname}/*.h"
            "${dirname}/*.cpp"
            )
    message("${testname}.cpp | ${dir_src}")
    add_executable("${testname}.cpp" ${dir_src})
endforeach (testsourcefile ${APP_SOURCES})

Debonair

Answer 5

So Why not use powershell to create the list of source files for you. Take a look at this script

param (
    [Parameter(Mandatory=$True)]
    [string]$root 
)

if (-not (Test-Path  -Path $root)) {    
throw "Error directory does not exist"
}

#get the full path of the root
$rootDir = get-item -Path $root
$fp=$rootDir.FullName;


$files = Get-ChildItem -Path $root -Recurse -File | 
         Where-Object { ".cpp",".cxx",".cc",".h" -contains $_.Extension} | 
         Foreach {$_.FullName.replace("${fp}\","").replace("\","/")}

$CMakeExpr = "set(SOURCES "

foreach($file in $files){

    $CMakeExpr+= """$file"" " ;
}
$CMakeExpr+=")"
return $CMakeExpr;

Suppose you have a folder with this structure

C:\Workspace\A
--a.cpp
C:\Workspace\B 
--b.cpp

Now save this file as "generateSourceList.ps1" for example, and run the script as

~>./generateSourceList.ps1 -root "C:\Workspace" > out.txt

out.txt file will contain

set(SOURCE "A/a.cpp" "B/b.cpp")

Ainsley

Question is tagged cross-platform - powershell is not cross-platform. – Corrective

@Corrective powershell works on linux and and mac too – Congruence

I know. But there's no tag for powershell and "CMake" is already crossplatform. "Powershell" would be an additional dependency and is very seldom used on other OS than Windows. You might find even more C# developers on Linux than people with powershell. And the most important fact is the question: "Is it possible to do it in directly in CMake? How?" – Corrective

Powershell absolutely is cross platform. – Amphibious

I don't think I will ever install Powershell on my Linux box or on my Mac. Thanks but no thanks. Need another solution. – Schoolhouse

I'd sooner use Bash for Linux and Mac then use something like Cygwin to make it work on windows. But even before that I'd use just CMake. – Shabbygenteel