When should I use GCC's -pipe option?

Question

The GCC 4.1.2 documentation has this to say about the -pipe option:

-pipe

Use pipes rather than temporary files for communication between the various stages of compilation. This fails to work on some systems where the assembler is unable to read from a pipe; but the GNU assembler has no trouble.

I assume I'd be able to tell from error message if my systems' assemblers didn't support pipes, so besides that issue, when does it matter whether I use that option? What factors should go into deciding to use it?

Permatron

I think you can be confident that if your GCC is configured using an assembler that does not support piped input, then GCC will either reject the -pipe option or ignore it (more likely the latter). – Magnifico

Answer 1

It doesn't usually make any difference

It has + and - considerations. Historically, running the compiler and assembler simultaneously would stress RAM resources.

Gcc is small by today's standards and -pipe adds a bit of multi-core accessible parallel execution.

But by the same token the CPU is so fast that it can create that temporary file and read it back without you even noticing. And since -pipe was never the default mode, it occasionally acts up a little. A single developer will generally report not noticing the time difference.

Now, there are some large projects out there. You can check out a single tree that will build all of Firefox, or NetBSD, or something like that, something that is really big. Something that includes all of X, say, as a minor subsystem component. You may or may not notice a difference when the job involves millions of lines of code in thousands and thousands of C files. As I'm sure you know, people normally work on only a small part of something like this at one time. But if you are a release engineer or working on a build server, or changing something in stdio.h, you may well want to build the whole system to see if you broke anything. And now, every drop of performance probably counts...

Herbal

the CPU is so fast that it can create that temporary file and read it back without you even noticing the statement is wrong. The bottleneck here isn't a CPU, but rather I/O of a drive. Imagine that you're compiling for some reason a big project on a slow USB stick: you going to be sick of waiting all the read/write operations. Of course GNU/Linux caching a files instead of actual write. But this isn't GCC's merit. So Do we have to use -pipe? yes, it is better to use. – Examine

@angel : I suspect that such i/o delay won't even be in the critical loop for modern systems : add the effect of memory cache, and as long as you have enough memory for recent files, all real-time operations will happen there, with "real write" on the device happening later on, asynchronously. – Groin

@angel, because the OS has a large RAM page-cache available and especially when /tmp is mounted (as it should be) either on RAM or without full recovery write barriers, the actual overhead on I/O remains CPU in this case. – Herbal

when using large C++ templates ( like CGAL inside of OpenSCAD ) i have found that preventing the use of pipe, and instead asking it to use the assembler on a file, can prevent the machine from running out of RAM and hitting swapspace. This is on machines with relatively low RAM like 512MB or 1G, also depending if -g and -O2 are being used – Helms

Answer 2

In our experience with a medium-sized project, adding -pipe made no discernible difference in build times. We ran into a couple of problems with it (sometimes failing to delete intermediate files if an error was encountered, IIRC), and so since it wasn't gaining us anything, we quit using it rather than trying to troubleshoot those problems.

Jack

+1 for evidence based answer, rather than "it doesn't use disk, so it's probably better..." – Politesse

And see @peterkarasev for evidence that -pipe does in fact make a positive difference when you're working in a distributed filesystem. – Hedwig

Answer 3

Trying this out now, it looks to be moderately faster to build when the source / build destinations are on NFS (linux network). Memory usage is high though. If you never fill the RAM and have source on NFS, seems like a win with -pipe.

Secor

Answer 4

Honestly there is very little reason to not use it. -pipe will only use a tad more ram, which if this box is building code, I'd assume has a decent amount. It can significantly improve build time if your system is using a more conservative filesystem that writes and then deletes all the temporary files along the way (ext3, for example.)

Magnify

You're making an incorrect assumption - machines that build code often have very little memory. Think about a Raspberry Pi, for example. – Hedwig

@JamesMoore you cross-compile when you encounter constrained systems rather than deal with memory problems on the target platform, right? – Prurigo

Answer 5

One advantage is that with -pipe will the compiler interact less with a file system. Even when it is a ram disk does the data still need to go through the block I/O and file system layers when using temporary files whereas with a pipe it becomes a bit more direct.

With files does the compiler first need to finish writing before it can call the assembler. Another advantage of pipes is that both, the compiler and assembler, can run at the same time and it is making a bit more use of SMP architectures. Especially when the compiler needs to wait for the data from the source file (because of blocking I/O calls) can the operating system give the assembler full CPU time and let it do its job faster.

Lipp

Answer 6

From a hardware point of view I guess you would use -pipe to preserve the lifetime of your hard drive.

Prejudge

I strongly believe Linux doesn't actually SAVE the file to the hard-drive, but just puts it in cache in RAM. – Coparcenary