Choosing between Scons and Waf in Large Projects

Question

We are thinking about converting a really large project from using GNU Make to some more modern build tool. My current suggestion is to use SCons or Waf.

Currently:

• Build times are around 15 minutes.
• Around 100 developers.
• About 10 percent of code is C/C++/Fortran rest is Ada (using gnatmake).

Potential hopes/gains on improvements are

• Shared Compiler Cache to cut down build times and requires disk space
• Easier maintenance

Does SCons scale well for this task? I've seen comments on it not scaling aswell as Waf. Those are however a couple of years old. Have scons gained in performanced the last years? If not, what is the reason for its bad performance compared to Waf.

Ceremony

You can use ccache to add a compiler cache without using scons or waf. – Dall

It's probably worth prototyping both ways. Skip any subprojects or files or options or platforms that require complicated exceptions and just put together as much of the basic skeleton as you can, and compare the performance, then try adding in one of the complicated bits to make sure it's as easy as you expect. – Elishaelision

I just want to add scons and waf have a are different inside. I've heard that waf is based on scons ideas but after introducing to waf i've notice that waf have many differences from scons and more complicated. – Prohibitionist

@Torsten: I pretty much agree with your overall sentiment, but a few extra details: From what I understand, waf started as a fork of scons, but they ended up first rewriting half the guts while trying to preserve the same external behavior, then abandoning that goal and rewriting things again. I'm not sure that the end result is really more complicated, but it is definitely harder to learn (especially since it keeps changing, there's less documentation, and there are fewer people to help). – Elishaelision

@Elishaelision +1, waf is certainly quite difficult, if you want to do advanced stuff - like creating waf tools. However, I find the documentation pretty good for stuff of moderate complexity. The waf book was sufficient for me, for the most part. – Balmy

@Elishaelision you know about the waf book which is kept up to date and covers a lot of topics. Beyond that they got a pretty responsive google usergroup. I formyself would go with waf, though I have to damit I only have experience with projects with about 10 developers. – Doall

No one seems to know when/if scons will work in Python3. – Underbodice

Answer 1

I have been developing a tool chain for our company that is built around waf. It targets Fedora, Ubuntu, Arch, Windows, Mac OSX and will be rolled out to our embedded devices doing cross-compilation on various hosts.

We have found the way that waf allows contained extensibility through the tools, features and other methods has made it incredibly easy to customise and extend for our projects.

Personally, I think it is brilliant and find it nicely abstracts the interfaces to different tools that are integrated.

Unfortunately, I have no in-depth experience with Scons but lots with GNU Make/Autotools. Our decision to go with waf after evaluating build tools was that we needed something that worked well everywhere which made our build tool being backed by python and that it was fast. I based my decision on these results and went from there.

Menswear

The benchmarks are interesting, but not very current. Scons 1.2.0 is from 2008 and Waf 1.5.7 is from 2009. – Pubes

@Demyn, yes they are old but the only ones I could find and I wasn't interested in performing my own benchmarks. waf has worked out brilliantly for our uses. The extension mechanisms it provides are excellent. It's odd distribution method actually works well for us as it is self-unpacking. – Menswear

Answer 2

In the past, SCons wasnt as performant, but lots of improvements have been added since then.

I like both options and had to make the same decision about 6 months ago. I went with SCons since it appears to have a larger user and support base.

Here is a helpful link that compares SCons to other build tools.

Conventionality

+1. I mostly agree with this. The problem with waf is that the API is unstable, and every non-KDE project that switches to it switches back off, so there's really no userbase. (It's also just as verbose as scons, and with less documentation.) I'd say that if your project doesn't hit the scons brick wall, where huge projects suddenly take 17x longer for each file you add, you're better of sticking with scons; otherwise, waf is worth looking at. – Elishaelision

well, we are using waf for Fortran projects and it works quite nice, and there is no connection to KDE what so ever. – Clericals

Answer 3

I personally prefer Waf because it's more flexible and doesn't have the variant directory issue.

Waf

Pros:

• Separate variant directory; you don't clutter your source folder with object files (SCons also has this, but it's not on by default and takes a few tries to get working)
• Very flexible
• Automatic dependency sorting
• Works on lots of Python versions(CPython 2, CPython 3, Jython, and PyPy)
• You distribute it with your application, so users just need Python

Cons:

• A pain to extend
• Horribly underdocumented(although the examples help)
• Doesn't differentiate between GCC and Clang well(not sure if SCons has that problem too)

SCons

Pros:

• Much simpler than Waf
• Easier to extend than Waf(see here)
• Somewhat better documented

Cons:

• Scalability issues (Note that SCons isn't actually quite that bad, buy it still gets a tad slower as size increases)
• Ugly (this is 100% personal opinion)

Bottom line

It depends on what you're looking for. In general, Waf seems very good at managing large projects(and not just because of speed), but, if you need to extend it, look elsewhere. On the other hand, SCons is much easier to use.

If you decide to go with Waf, just post your problems to the mailing list.

Eckhart

SCons also supports separate src/build directories. – Triboluminescence

@JonathonReinhart Yeah, but it takes a few tries to get variant dirs right, unlike Waf, which has it out-of-the-box. Edited the post anyway. – Eckhart

Well you can always force use the compiler via bld.env['CC'] = 'clang' and bld.env['CXX']='clang++' if the containing directory is in your $PATH – Doall