Isolating gitsubmodule projects in CMake

Question

I'm trying to manage my C++ project dependencies using CMake and gitsubmodules. I'm following the layout described here: http://foonathan.net/blog/2016/07/07/cmake-dependency-handling.html and it's worked really well for me on smaller projects. But I've started to use it on much larger projects and I'm hitting some issue with CMake.

My current setup

All my external build dependencies are in a contrib/ subfolder inside my main project. Each is a submodule and has its own separate directory.

/contrib
- /eigen
- /curl
- /leapserial
- /zlib
- /opencv
etc.

The contrib/CMakeListst.txt simply initializes the submodule and adds the subdirectory for each external dependency

# EIGEN
execute_process(COMMAND git submodule update --recursive --init -- eigen
 WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
# options..
add_subdirectory(eigen EXCLUDE_FROM_ALL)

# CURL
execute_process(COMMAND git submodule update --recursive --init -- curl
 WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
# Initialize cache with CMake build options..
add_subdirectory(curl EXCLUDE_FROM_ALL)

# LEAP SERIAL
execute_process(COMMAND git submodule update --recursive --init -- leapserial
 WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
# Initialize cache with CMake build options..
add_subdirectory(leapserial EXCLUDE_FROM_ALL)

# ZLIB
execute_process(COMMAND git submodule update --recursive --init -- zlib
 WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
# Initialize cache with CMake build options..
add_subdirectory(zlib EXCLUDE_FROM_ALL)

# OPENCV
execute_process(COMMAND git submodule update --recursive --init -- opencv
  WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
# Initialize cache with CMake build options..
add_subdirectory(opencv EXCLUDE_FROM_ALL)

This setup has worked fantastically for me:

1. It's system/packagemanager independent. You don't need to install any libraries to get started developing
2. I can maintain the exact versions of my dependencies by setting the submodule to a particular commit. There are no surprises with some external library breaking your build
3. Adding the libraries to my build in the root CMakeListst.txt is trivial. Since I have the target available I just have something like

add_executable(someProjectImWorkingOn
  ${CMAKE_CURRENT_SOURCE_DIR}/src/main.cpp )
target_link_libraries(someProjectImWorkingOn
  opencv_world
  eigen
  zlib
  etc.)

4. when you hook up an existing library target to your own target executable/library CMake will automatically (through the target interface) add include directories to your target and add any other necessary options the library target requires for it to be used
5. I can pick a toolchain/compiler-option/build-type in the root CMakeLists.txt and it'll propogate to all the subprojects (I need to build for multiple systems. So this is a big big deal)
6. Since it's all in one "mega-project" it makes it very easy to hook up to rtags/kdevelop/clion to navigate not on your own code, but also the library code

Some issues that I can't resolved:

1

Subdirectories will define targets with the same name. In the example I gave, both Eigen OpenCV as well as another library define an 'uninstall' target

I tried to update the

add_subdirectory(blah)

to

add_subdirectory(blah EXCLUDE_FROM_ALL)

but this doesn't fix the issue for some reason

Enabling the variable ALLOW_DUPLICATE_CUSTOM_TARGETS kinda works.. but this is a hack, only work with Make files, and the libraries are essentially still "mixing" so it's still an issue

2

The second issue came up in LeapSerial but illustrates a bigger issue. The project no longer knows it's own name. LeapSerial tried to determine the version of LeapSerial, but when it asks for the project version it's getting the root project version. Ie. when cmake code in a subproject asks for "what project am I in" it's getting the root project, and not the immediate project it's in.

So again, the parent "namespace"s are leaking everywhere. This is bound to create more and more issues down the line. I need to the submodules to be self-contained

Is there are a cleaner solution?

ExternalProjectAdd might solve some of these problems, but has a lot more issues of its own. It's a real non-starter b/c it doesn't do most of what I've listed. The central issue is that it doesn't expose the sub-project's targets - and just vomits back variables that you then have to juggle

Wilkison

Whoever wrote that tutorial definitely needs to read "The more times you use the word "simply" in your instructions, the more I suspect you don’t know what that word means" article. – Viewless

ExternalProject_Add is more convenient than you think. E.g. git and revisions are already incorporated into it. As for "it doesn't expose the sub-project's targets", being combined with execute_process (that is, 3d-party libraries are built on configure step), it allows to use find_package for external libraries. As for add_subdirectory() approach, targets and variables conllision is unavoidable. CMake doesn't provide neither variable nor target namespaces. – Rattail

right, but find_package just vomits back some variables that you then have to manage. It doesn't handle everything automatically like when you use target_link_libraries. nor will it cleanly manage toolchains and ties ins with IDEs – Wilkison

I guess if there are no namespaces then I guess I'm screwed :( CMake isn't a mature enough tool for easy development yet – Wilkison

I finally sorta resolved this by using the Hunter package manager. It's 100% CMake based and works very well for my needs – Wilkison

Answer 1

As the asker said in the comments, they resolved their issue by using the Hunter package manager. The rest of this answer is about actually answering the question as posed.

Concerning your first issue with target name clashes when using add_subdirectory-based approaches to using dependencies, a very similar (or essentially the same?) question has also since been asked here: How to avoid namespace collision when using CMake FetchContent?. When the clashes are between targets from different project dependencies, there's nothing you can do right now except politely ask the project maintainers to consider modifying their non-import/export target names to be namespaced like.

For example, for a library target, that might look like:

add_library(projectname_targetnamepart)
add_library("importexportnamespacename::targetnamepart" ALIAS projectname_targetnamepart)
set_target_properties(projectname_targetnamepart PROPERTIES EXPORT_NAME targetnamepart)
set_target_properties(projectname_targetnamepart PROPERTIES OUTPUT_NAME targetnamepart)
install(TARGETS projectname_targetnamepart EXPORT projectname_targets ...)
install(EXPORT projectname_targets NAMESPACE "importexportnamespacename::" ...)

There's a Kitware issue ticket by Craig Scott proposing a CMake feature for Project-level namespaces. Here's an excerpt:

A common problem when pulling in subprojects using add_subdirectory() is that target names must be unique, but different subprojects might try to use the same target name, which results in a name clash. This issue proposes to introduce the concept of a project namespace to address this and related name uniqueness problems (this is the primary goal of this issue).

Sometimes the upstream maintainer will just decline to support the add_subdirectory / FetchContent use-case. That's the case with OpenCV, as shown in this issue ticket (#16896). As for eigen, there's an open ticket that hasn't had any activity in a while (#1892).

Concerning your second issue, there's not enough detail in your question post to confidently troubleshoot. What is LeapSerial? Are you referring to the leapmotion/leapserial GitHub repo? What version and what commit are you referring to? At the latest commit before the time of your question post, 41515db, it's not immediately obvious what's wrong.

Variables in CMake are scoped by directory, so even if a project is added by add_subdirectory and doesn't used the <PROJECT-NAME>_VERSION variable and instead uses the more general PROJECT_VERSION variable, it should be okay. It just shouldn't attempt to use the CMAKE_PROJECT_VERSION variable to get its own version, since that one is fixed to refer to the top-level project's version.

Snaggy