SCons code generation and VariantDir

Question

I would like SCons to generate some source files for me in my src/ directory, and then build them as any other source file in my build directory build/variantX.

This is my SCons file:

import SCons

def my_builder(env, target, source):
    # do stuff
    pass

env = Environment()
env.VariantDir('build/variant1/', 'src', duplicate=0)
env.Command('src/foobar.cc', 'src/foobar.input', action=my_builder)
env.Program('bin/test', [
    'build/variant1/foobar.cc',
    'build/variant1/test.cc',
    ])

This errors with the following message:

Source src/foobar.cc not found, needed by target build/variant1/foobar.o

which I don't think is correct, considering that I am indeed providing a command to build src/foobar.cc.

Now, I tried a few workarounds:

• if I replace build/variant1/foobar.cc in Program with src/foobar.cc, it does work, but obviously foobar.o gets created in src/ rather than build/variant1

• if I replace src/foobar.cc in Command with build/variant1/foobar.cc, it does work, but I would like the code to be generated in src/; (also because things like relative paths in include directories won't work unless duplicate=1)

• if duplicate=1, I get a similar error message, but this time mentioning the variant directory:

  Source build/variant1/foobar.cc not found, needed by target build/variant1/foobar.o

Is there a way around this? Is it a limitation/bug in SCons, or is there a fundamental misunderstanding on my side?

Anchorite

Did you ever find an answer to this problem? I'm facing something very similar. – Luxuriance

Answer 1

I would suggest creating an explicit dependency between the Command() and Program() calls as follows:

target1 = env.Command('src/foobar.cc', 'src/foobar.input', action=my_builder)
target2 = env.Program('bin/test', [
                      'build/variant1/foobar.cc',
                      'build/variant1/test.cc',
                      ])
Depends(target2, target1)
# This should work too
# Depends(target2, "src/foobar.cc")

Or you could specify the target from the Command() as part of the source for Program() as follows:

target1 = env.Command('src/foobar.cc', 'src/foobar.input', action=my_builder)
env.Program('bin/test', [
            target1,
            'build/variant1/test.cc',
            ])

I havent tested this, so Im not sure how it will work in conjunction with the call to VariantDir()

Here is some extra info regarding generating source code with SCons.

Halicarnassus

Hi Brady, thanks for your help. Your first idea was nice indeed but it doesn't seem to have any effect. The second one seems to be effectively the same as my first workaround: it works but it builds it in the source directory rather than the variant one. The link looks interesting, I am now going through it. – Anchorite

@Anchorite Im really surprised the Depends() function didnt work. Try using the actual file name instead, I'll update the answer. – Halicarnassus

--tree=all tells me that this results in bin/test depending on the "correct" src/foobar.cc (the one that in turn depends on src/foobar.input), but build/variant1/foobar.o still depends on a "dangling" src/foobar.cc, which doesn't depend on anything in turn and (open fiddling with the scons source code) doesn't even seem to have any builder attached to it. I'm really starting to think that this is a Scons bug. – Anchorite

I think VariantDir fundamentally breaks something in the dependency tree. I managed to come up with a very simple broken test case, that doesn't involve code generation at all. Have a look here if you're interested: scons.tigris.org/issues/show_bug.cgi?id=2908 – Anchorite

@Anchorite Another alternative to directly calling the VariantDir() function would be to create a hierarchical build, whereby you use the SConscript() function and use the variant_dir parameter. This is more straightforward and doesnt have the strange side-effects of the VariantDir() call. I guess it depends on the structure of your project though. – Halicarnassus

Answer 2

I know it has been a while, but i hit the same wall. With minor modifications to the 'test case' and solution (see below) the code is:

import SCons

env = Environment()
env.VariantDir('build/variant1/', 'src', duplicate=0)
env.Command('src/foobar.cc', 'src/foobar.input', action="cp src/foobar.input src/foobar.cc", shell=True )
env.Depends("build/variant1/foobar.cc", "src/foobar.cc")
env.Program('bin/test', [
'build/variant1/foobar.cc',
])

The added 'env.Depends' on the 'variantdir-source' to 'generated-source' is the key. No idea why this is needed. I would call it a bug, but i guess its featured (based on the bug feedback you got .. )

Cheers,

Soulful

This solved it for me. Seems to be a bug, scons should be able to figure out that variant version of the file is a target related to the generated source. – Twitter

Answer 3

FYI I found another solution to this problem that works when using SConscript(variant_dir=...). In my project I don't have a separate 'src' directory, I do have hierarchical SConscript files, and ideally I'd rather not name the build directory everywhere because it depends on the target being built.

The key is (a) depending directly on the File rather than a string filename, which seems to confuse the dependency tree [this solves the dependency issue, per above], and (b) explicitly creating the Object with a name that causes it to be generated in the build directory, rather than the source directory [solving the .o-in-src/ problem].

Here's what I have that demonstrates this working,

SConstruct:

env = Environment()
Export('env')
SConscript('src/SConscript', variant_dir='build/variant1', duplicate=False)

src/SConscript:

Import('env')
s = env.Command('#/src/foobar.cc', 'foobar.input', action="cp $SOURCES $TARGETS", shell=True )
o = Object(target="foobar.o", source=s)
env.Program('#/bin/test', [o])

Building gives me,

$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
cp src/foobar.input src/foobar.cc
g++ -o build/variant1/foobar.o -c src/foobar.cc
g++ -o bin/test build/variant1/foobar.o
scons: done building targets.

The primary downside is that you have to 'root' the target source file from the root of the tree rather than giving a relative name, to get it to generate in the source directory rather than the build/variant directory.

Any suggestions on how to do this more elegantly would be great! I currently have it wrapped in a Builder that handles the above automatically, but I still have to supply an absolute filename, which is inconvenient.

Beerbohm