Avoiding extra ExternalProject downloads

Question

Let's say I have the following project setup with these dependencies:

MainProject
├─ Dependency_1
│  └─ Dependency_2
└─ Dependency_2

These dependencies are handled in MainProject and Dependency_1 with ExternalProject.

The problem is Dependency_2 will be downloaded twice: Dependency_1 will download a copy for itself, and MainProject will download a copy for itself.

This doesn't make for an efficient build process, is there a way where I can download Dependency_2 once for both projects?

---

Is has been suggested that this question is a duplicate of this one. That question slightly varies from mine, in that I cannot assume these libraries will be installed to the host system with ExternalProject. I would also like a CMake only solution, to which that question did not require.

Lindsy

Probably this project should be handy: github.com/ruslo/hunter – Animalist

Your question is probably a duplicate of #38060159 – Animalist

@Animalist I would like a CMake only solution. Also, that question mentions already installed libraries, the dependencies in my question shouldn't be assumed to have been installed on the host system. – Lindsy

What are your current ExternalProject commands? I would first of all try to put a common DOWNLOAD_DIR and BUILD_DIR – Puerperium

@Puerperium Here is a list of the ones I use for a project of mine. Imagine PortAudio is like Dependency_2, tritium is like Dependency_1, and Khronos is like MainProject. – Lindsy

@Puerperium What do you mean "encoded"? Like where I use it in my projects .c files? Or the main CMakeLists.txt – Lindsy

Where is tritium declaring its dependency to PortAudio? – Puerperium

@Puerperium Ah, I understand now. Here is where that ExternalProject use is. – Lindsy

So, the first thing I would try would be to have a common source and build directory declared for the 2 dependencies to PortAudio. First try with some hardcoded path. If it works, then we can think what would be the most elegant way to do this. – Puerperium

@Puerperium Any recommendations for a hard-coded path should I choose so this works on both Windows and Unix systems? – Lindsy

@Puerperium Sadly, I cannot test the concept right now since I'm at work. I'll do my best to try this when I get home in several hours. – Lindsy

I wonder if the only way would be for the main CMakeLists.txt to set an environment variable with a common root path (e.g. "${CMAKE_SOURCE_DIR}/externalProjects" and "${CMAKE_BINARY_DIR}/externalProjects" to be used as root folders for the download and the builds of the dependencies). You can set (and use) it in Khronos, and read this value from tritium. See docs set env. I don't know if a cache variable set from Khronos would be visible in tritium... – Puerperium

@Lindsy Any feedback? – Puerperium

Answer 1

From the main CMakeLists.txt, set an environment variable containing a common root path for downloading and building external projects, for example:

set (ENV EXTERNAL_PROJ_DOWNLOAD_DIR "${CMAKE_SOURCE_DIR}/externalProjects")

to be used as root folders for the download and the builds of the dependencies. You can set (and use) it in your main project, and read this value from within your first dependency (the one that also depends to your second dependency).

Seen it in practice applied to the project linked in your comments, you'll set EXTERNAL_PROJ_DOWNLOAD_DIR IN Khronos, and then to link to PortAudio in both Khronos and tritium projects you will have:

find_package(PortAudio)
if (${PORTAUDIO_FOUND})
    include_directories(${PORTAUDIO_INCLUDE_DIRS})
else ()
    ExternalProject_Add(
        PortAudio
        GIT_REPOSITORY      "https://github.com/syb0rg/PortAudio2.git"
        SOURCE_DIR          "$ENV{EXTERNAL_PROJ_DOWNLOAD_DIR}/PortAudio"
        UPDATE_COMMAND      ""
        INSTALL_COMMAND     ""
        BUILD_IN_SOURCE     ON
        LOG_DOWNLOAD        ON
        LOG_UPDATE          ON
        LOG_CONFIGURE       ON
        LOG_BUILD           ON
        LOG_TEST            ON
        LOG_INSTALL         ON
    )
    ExternalProject_Get_Property(PortAudio SOURCE_DIR)
    ExternalProject_Get_Property(PortAudio BINARY_DIR)
    set(PORTAUDIO_SOURCE_DIR ${SOURCE_DIR})
    set(PORTAUDIO_BINARY_DIR ${BINARY_DIR})
    set(PORTAUDIO_LIBRARIES ${PORTAUDIO_SOURCE_DIR}/libportaudio_static.a)
    set(DEPENDENCIES ${DEPENDENCIES} PortAudio)
    include_directories(${PORTAUDIO_SOURCE_DIR}/include)
endif ()
SET(LIBS ${LIBS} ${PORTAUDIO_LIBRARIES})

You could also use set (ENV EXTERNAL_PROJ_BINARY_DIR "${CMAKE_BINARY_DIR}/externalProjects") if you wanted to activate the out of source build.

I suggest to use an environment variable because I don't know if a cache variable set from Khronos would be visible in tritium...

See documentation for set and env.

Puerperium