add_custom_command is not generating a target

Question

Perhaps this is impossible and I'm misreading the cmake 3.2 documentation, but I though creating a custom command would create a custom "target" in the Makefile so that I could build the target by invoking the name of the output file. The CMake docs says:

In makefile terms this creates a new target in the following form:

 OUTPUT: MAIN_DEPENDENCY DEPENDS
    COMMAND

so I thought I could then run make OUTPUT. Perhaps the documentation is confusing CMake targets with Makefile targets?

For example,

 add_custom_command(OUTPUT foo_out
    COMMAND post_process foo_in > foo_out
    DEPENDS foo_in
 )

I would like to do

 make foo_out

and it will make foo_out. However, if I do this, I get

make: **** No rule to make target `foo_out`. Stop.

and sure enough, the word "foo_out" doesn't exist anywhere in any file in the cmake binary output directory. If I change it to this

add_custom_target(bar DEPENDS foo_out)
add_custom_command(OUTPUT foo_out COMMAND post_process foo_in > foo_out)

Then I can do

make bar

and I can do

make foo_in

but I still can't do

make foo_out

The problem with make bar is that it is unintuitive, as the actual file output is foo_out not bar.

How do I do this?

In my case, I need to run a special processing step to the standard executable target which inserts optional resources into the ELF file. I would like the ability to have both executables as Makefile targets, so I can build the naked ELF executable as well as the resource-injected ELF executable.

If I was writing a custom Makefile, this is trivial to do!

foo_in: foo.c
    $(CC) $< -o $@

foo_out: foo_in
    post_process $< > $@   

And I can do make foo_in and make foo_out.

Volcanic

I think you need to specify the OUTPUT foo as an input to any other target (not necessarily a custom target). The add_custom_command() documentation says "This defines a command to generate specified OUTPUT file(s). A target created in the same directory (CMakeLists.txt file) that specifies any output of the custom command as a source file is given a rule to generate the file using the command at build time." – Brenan

@Brenan - In my example bar target, I still can't build foo directly (if I do make foo nothing happens) even though bar depends on foo. – Volcanic

Thanks for the additional details. You can't do something like add_custom_target(foo_out DEPENDS foo_out) in CMake because it tries to match the dependencies to target names. And after doing some testing with a GNU Makefile generator I see that the actual build rule for foo_out as in your example is generated only into ´CMakeFiles\bar.dir\build.make´ and is not made visible in the main Makefile. So I tired: add_custom_target(foo_out COMMAND post_process foo_in > foo_out) with set_property(DIRECTORY APPEND PROPERTY ADDITIONAL_MAKE_CLEAN_FILES foo_out). Not nice, but the names match ... – Brenan

@Brenan I just tried and for me with Unix Makefile generator the target is found in the directory where I added add_costum_command and in the build root directory. Nowhere is. The documenation is lacking any details as far as I see. – Heidyheifer

Answer 1

add_custom_command does not create a new target. You have to define targets explicitly by add_executable, add_library or add_custom_target in order to make them visible to make.

If you have to fix things up for deployment, you could

1. use the install command (somewhere in your CMakeLists.txt) like this:

install(SCRIPT <dir>/post_install.cmake)

to store commands which are executed only when you run make install in a separate .cmake file. Or if the install target is already reserved for other things or you have more complex stuff going on:

2. manually define a deploy target. Once you got that, you can create a custom post-build command which is only executed when you explicitly run make on your deploy target. This allows you to execute commands through a separate target.

In your CMakeLists.txt this could look like:

cmake_minimum_required(VERSION 3.0)

add_executable("App" <sources>)

# option 1: do deployment stuff only when installing
install(SCRIPT <dir>/post_install.cmake)

# option 2: define a deploy target and add post-build commands
add_custom_target("deploy")
add_custom_command(TARGET "deploy" POST_BUILD <some command>)

Both approaches allow you to separate dev builds from expensive ready-to-deploy builds (if I understand correctly, that's the goal here). I would recommend option 1 since it's just cleaner.

Hope this helps!

Gariepy

Okay, but still I don't see why install is not the thing you want here. It does exactly what you want: provide a target to make your product ready for deplyoment (fix lib/resource paths, copying stuff around) while separating build configurations for development and deployment builds. – Gariepy

make install is effectively the same as make bar. But I want to do is make foo_out. If I had 100 primary files and wanted to generate 100 secondary targets, I don't want to have to create 100 dummy targets (like bar or install) to do this. – Volcanic

what if I want to use if/else to decide whether to execute a command in add_custom_target or add_custom_command ? – Peria

Answer 2

Documentation Unclear

CMake's documentation is unclear here. The Makefiles generators of CMake do create the source file make rules in sub Makefiles which are not visible in the main Makefile. In the main Makefile you will find only the PHONY rules for your CMake targets. The only exception I know of is the Ninja Makefiles generator which puts all build rules into single file.

Translating Post-Processing Steps into CMake

From my experience - if post_process is a script - you should probably think about rewriting your post-processing steps with/inside the CMake scripts, because CMake should know about all the file dependencies and variables used for post-processing (it then will e.g. handle all the necessary re-build or clean-up steps for you).

Here is a simplified/modified version of what I do:

function(my_add_elf _target)

    set(_source_list ${ARGN})
    add_executable(${_target}_in ${_source_list})

    set_target_properties(
        ${_target}_in
        PROPERTIES
            POSITION_INDEPENDENT_CODE   0
            SUFFIX                      .elf
    )

    add_custom_command(
        OUTPUT ${_target}_step1.elf
        COMMAND some_conversion_cmd $<TARGET_FILE:${_target}_in> > ${_target}_step1.elf
        DEPENDS ${_target}_in
    )

    add_custom_target(
        ${_target}_step1 
        DEPENDS 
            ${_target}_step1.elf
    )

    add_custom_command(
        OUTPUT ${_target}_out.elf
        COMMAND final_post_process_cmd ${_target}_step1.elf > ${_target}_out.elf
        DEPENDS ${_target}_step1
    )

    add_custom_target(
        ${_target}_out 
        DEPENDS 
            ${_target}_out.elf
    )

    # alias / PHONY target
    add_custom_target(${_target} DEPENDS ${_target}_out)

endfunction(my_add_elf)

and then call

my_add_elf(foo foo.c)

It's only an example, but I hope it gives the idea: you could call make foo for the final ELF output, make foo_in or make foo_step1 for one of the other steps. And I think all steps are transparent for the user and CMake.

Can't give your Target the same name as one of the Outputs

When you're trying to give a custom target the same name as one of its outputs e.g. like this:

add_executable(foo_in foo.c)
add_custom_command(
    OUTPUT foo_out
    COMMAND post_process foo_in > foo_out
    DEPENDS foo_in
)
add_custom_target(foo_out DEPENDS foo_out)

You end-up with invalid make files. I've raised an issue about this in the hope that there could be a possible solution by extending CMake itself and got the following reply:

CMake is not intended to produce specific content in a Makefile. Top-level target names created by add_custom_target are always logical (i.e. phony) names. It is simply not allowed to have a file of the same name.

Posible Workarounds

So there are some workarounds, but they all have one or the other disadvantage.

1. Shortest Version:

macro(my_add_elf_we _target)
    add_executable(${_target}_in ${ARGN})
    add_custom_target(
        ${_target}_out 
        COMMAND post_process $<TARGET_FILE:${_target}_in> > ${_target}_out
        DEPENDS ${_target}_in
    ) 
    set_property(DIRECTORY APPEND PROPERTY ADDITIONAL_MAKE_CLEAN_FILES ${_target}_out)
endmacro(my_add_elf_we)

You can't declare OUTPUTs in the add_custom_target() itself, but in this case you don't want to (because you don't want to have any naming confusions). But if you don't declare any outputs:

• The target will always considered out-of-date
• You need to add the "invisible" outputs the clean build rule

2. Force Output Name Version

Here is a version of the above macro that forces target and output names to given values:

macro(my_add_elf_in_out _target_in _target_out)
    add_executable(${_target_in} ${ARGN})
    set_target_properties(
        ${_target_in}
        PROPERTIES
            SUFFIX          ""
            OUTPUT_NAME     "${_target_in}"
    )
    add_custom_target(
        ${_target_out}
        COMMAND post_process ${_target_in} > ${_target_out}
        DEPENDS ${_target_in}
    ) 
    set_property(DIRECTORY APPEND PROPERTY ADDITIONAL_MAKE_CLEAN_FILES ${_target_out})
endmacro(my_add_elf_in_out)

You call it with:

my_add_elf_in_out(foo_in.elf foo_out.elf foo.c)

3. Object Libary Version

The following version uses object libraries, but the system will not reuse the foo_in target linkage:

macro(my_add_elf_obj_in_out _target_in _target_out)

    add_library(${_target_in}_obj OBJECT ${ARGN})

    add_executable(${_target_in} $<TARGET_OBJECTS:${_target_in}_obj>)
    set_target_properties(
        ${_target_in}
        PROPERTIES
            SUFFIX          ""
            OUTPUT_NAME     "${_target_in}"
    )

    add_executable(${_target_out} $<TARGET_OBJECTS:${_target_in}_obj>)
    set_target_properties(
        ${_target_out}
        PROPERTIES
            SUFFIX              ""
            OUTPUT_NAME         "${_target_out}"
            EXCLUDE_FROM_ALL    1
    )
    add_custom_command(
        TARGET ${_target_out}
        POST_BUILD
        COMMAND post_process ${_target_in} > ${_target_out}
    )

endmacro(my_add_elf_obj_in_out)

4. Last and Final Version

And one final version that definitely works only for with Makefile generators and that got me posting the issue at CMake's bug tracker:

macro(my_add_elf_ext_in_out _target_in _target_out)

    add_executable(${_target_in} ${ARGN})
    set_target_properties(
        ${_target_in}
        PROPERTIES
            SUFFIX          ""
            OUTPUT_NAME     "${_target_in}"
    )
    add_executable(${_target_out} NotExisting.c)
    set_source_files_properties(
        NotExisting.c
        PROPERTIES
            GENERATED           1
            HEADER_FILE_ONLY    1
    )
    set_target_properties(
        ${_target_out}
        PROPERTIES
            SUFFIX              ""
            OUTPUT_NAME         "${_target_out}"
            RULE_LAUNCH_LINK    "# "
    )
    add_custom_command(
        TARGET ${_target_out}
        POST_BUILD
        COMMAND post_process ${_target_in} > ${_target_out}
    )
    add_dependencies(${_target_out} ${_target_in})

endmacro(my_add_elf_ext_in_out)

Some references

• CMake add_custom_command/_target in different directories for cross-compilation
• CMake: how do i depend on output from a custom target?
• cmake add_custom_command
• How do I use CMake to build LaTeX documents?

Brenan

thanks but this still doesn't solve the problem. Using your example, I need to type make foo to generate foo.elf. I would like to do make foo.elf, that is, use the actual file name as the target. But thanks for the ADDITIONAL_MAKE_CLEAN_FILES tip, I wasn't aware of that. – Volcanic

Thanks for the feedback. I've tested my my_add_elf_we() macro version again (in the lower part of my answer), added some fixes and I think it now does exactly what you wanted. The output name and the make rule names are the same. In my example if you have my_add_elf_we(foo foo.c) in your CMakeLists.txtyou can call make foo_out and make foo_in. And you can change the namings of the targets/outputs or you could also add a PHONY target for just foo to the macro. – Brenan

I appreciate your help, but that still doesn't do what I want. I want to do make foo_out.elf and I want to see a file called foo_out.elf produced. I also want make help display foo_out.elf as a possible target. – Volcanic

I'm confident to find a solution. So I added another version of my macro my_add_elf_in_out() where the target and output names are forced to given values. – Brenan

edit #1 has the problem that the target is always remade (since it is a custom target). edit #2 - hmm, that's very clever, but maybe too clever for me. I think your idea of writing to the cmake developers is great! – Volcanic

Yes, I believe the right solution is to improve/fix this in CMake itself and submitted an issue to CMake's bug tracker: 0015614: add_custom_target/command: Can't give Target the same name as one of the Outputs – Brenan

Added response from CMake team and the object library version. – Brenan

Not much of a response. :( I don't see the reason why they distinguish between the standard targets (executable or library) which do allow the name of the target and the file output to be the same, and custom targets.I think they just don't want to fix it. – Volcanic

I know. But our request is now out there and I think we have to wait if we get more support from the community before we may push the point again. At the moment it seems like being just the two of us having this discussion. Maybe you could also add your thoughts to the issue/ticket in CMake's bug tracker. Otherwise I only see one of us actually doing the changes in CMake and see if the community would accept our contribution. – Brenan

Answer 3

Turning around the dependencies, and using the second signature of add_custom_command, this should work:

add_custom_target(foo_out DEPENDS foo_in)
add_custom_command(TARGET foo_out POST_BUILD COMMAND post_process foo_in > foo_out)

Note: Adding BYPRODUCTS foo_out will cause (for example) ninja to say

multiple rules generate foo_out. builds involving this target will not be correct; continuing anyway

Amadaamadas