Will printf still have a cost even if I redirect output to /dev/null?

Question

We have a daemon that contains a lot of print messages. Since we are working on an embedded device with a weak CPU and other constraint hardware, we want to minimize any kinds of costs (IO, CPU, etc..) of printf messages in our final version. (Users don't have a console)

My teammate and I have a disagreement. He thinks we can just redirect everything to /dev/null. It won't cost any IO so affections will be minimal. But I think it will still cost CPU and we better define a macro for printf so we can rewrite "printf" (maybe just return).

So I need some opinions about who is right. Will Linux be smart enough to optimize printf? I really doubt it.

Aprilaprile

Beware side effects: printf("%d", x=a+b); If you redirect to /dev/null side effects will happen; if you rewrite as a do nothing macro, side effects will be lost – Leannaleanne

Providing a myprintf(...) { return; } is probably what you want. You can then have a macro for printf forwarding to that method, preserving side effects yet not formatting any string or calling write – Tolu

put the printf stuff in a preprocessor block and turn it off for production – Celeriac

@pmg: Side-effects in a printf statement are evil. In code review, I'd definitely raise an issue over that. – Warms

I would take a step back. In a Linux daemon there are far better options then printf... Consider for example using syslog instead, if you set the log level at startup (ideally from an environment variable) and you can direct the logs to file or to another machine over the network trivially, and the log level allows you to turn off the logging of things you don't care about at relatively low cost in execution time. Even better if you do something like trapping a few signals you can arrange to change the log level at runtime without stopping, much less having to recompile the daemon. – Ehr

Related: #44613918 /dev/null just ignores the buffer. printf still has to do the work of writing to it. – Dubrovnik

Seems like you need a proper logging framework. At the minimum, one that supports lazy evaluation of the message. – Convolve

The answers are maybe a bit naive about the correct choice, though the information is correct. 1) the penalty for IO can be gigantic on weird embedded systems (and with pathological logging), so you may recover much of the cost despite the printf cost; 2) the cost may be so small that it's better just to redirect to /dev/null for simplicity and to reduce your workload as developers. – Petua

@DanMills You left out how using printf() for logging ties the deamon to the output file selected at startup - for the life of the daemon process. printf() output is also buffered, which means abnormal termination of a process leads to lost log entries - right when you need to see exactly what's going on. That's two more reasons printf() is a very, very, very bad logging mechanism. :-) – Brisling

@AndrewHenle True, and the fact that it is holding a file descriptor to a disk file open means that logrotate or whatever cannot drop the reference count to zero and free the disk space. You can unlink the filename, but until the daemon terminates the space used on disk will just keep on growing. Printf sucks as a logging mechanism, and odds are syslog is there in the system, so may as well use it. – Ehr

Your teammate is lacking in the most basic understanding of how computers, programs, operating systems, and libraries work. – Cupulate

'printf() output is also buffered' -- only if one is foolish enough not to flush it. – Cupulate

Answer 1

Pretty much.

When you redirect the stdout of the program to /dev/null, any call to printf(3) will still evaluate all the arguments, and the string formatting process will still take place before calling write(2), which writes the full formatted string to the standard output of the process. It's at the kernel level that the data isn't written to disk, but discarded by the handler associated with the special device /dev/null.

So at the very best, you won't bypass or evade the overhead of evaluating the arguments and passing them to printf, the string formatting job behind printf, and at least one system call to actually write the data, just by redirecting stdout to /dev/null. Well, that's a true difference on Linux. The implementation just returns the number of bytes you wanted to write (specified by the 3rd argument of your call to write(2)) and ignores everything else (see this answer). Depending on the amount of data you're writing, and the speed of the target device (disk or terminal), the difference in performance may vary a lot. On embedded systems, generally speaking, cutting off the disk write by redirecting to /dev/null can save quite some system resources for a non-trivial amount of written data.

Although in theory, the program could detect /dev/null and perform some optimizations within the restrictions of standards they comply to (ISO C and POSIX), based on general understanding of common implementations, they practically don't (i.e. I am unaware of any Unix or Linux system doing so).

The POSIX standard mandates writing to the standard output for any call to printf(3), so it's not standard-conforming to suppress the call to write(2) depending on the associated file descriptors. For more details about POSIX requirements, you can read Damon's answer. Oh, and a quick note: All Linux distros are practically POSIX-compliant, despite not being certified to be so.

Be aware that if you replace printf completely, some side effects may go wrong, for example printf("%d%n", a++, &b). If you really need to suppress the output depending on the program execution environment, consider setting a global flag and wrap up printf to check the flag before printing — it isn't going to slow down the program to an extent where the performance loss is visible, as a single condition check is much faster than calling printf and doing all the string formatting.

Gibbie

Answers like this should state they are based on general understanding of common implementations and not upon specific documentation. In theory, there is no reason a C implementation might not inspect stdout, learn it is /dev/null, and suppress printf calls that do not contain %n and whose return value is not used. We cannot really assert nobody has done this, and students ought to learn the provenance of information since an important part of engineering is knowing how you know something (is it specified in a standard, is it just assumed, is it provable, and so on). – Sung

@EricPostpischil Thanks for that! Very valuable information. – Gibbie

@EricPostpischil See Damon's answer for reference from the POSIX standard - supressing a write(2) call to write stdout is forbidden by the standard. – Gibbie

AFAIK, console output is pretty slow so the time saved by writting to /dev/null could be significant. It would be less effective if you were writting to file, though. – Zymolysis

@Zymolysis Writing to a terminal isn't necessarily slow, displaying it is (i.e. depends on terminal). – Gibbie

@Zymolysis There are cases where one would still care about the runtime costs. I've personally worked in situations where I found the need to have a debug print statement running in a tight loop for debugging, where the runtime cost of formatting the output with printf or sprintf would be excessive in a runtime environment. It's not clear if the OP is in such a situation, but it's a case I've found where the formatting costs alone are sufficient to matter. – Ultramicroscopic

@Gibbie So if no terminal physically connected to the system(by wire or tftp, etc..), then print function will have the same affections as redirecting to dev/null? Since they both have no displaying... – Aprilaprile

@MichaelPeng Depends on the handler associated with the terminal - usually it can't be as fast as /dev/null, but some programs create pseudo-terminals (/dev/pts/x) that may be faster than other terminals. – Gibbie

Who's right? Pretty Much. First name Pretty, second name Much. – Uncovenanted

LOL pretty joke Jasper @Uncovenanted – Aprilaprile

In theory, there is no reason a C implementation might not inspect stdout, learn it is /dev/null, and suppress printf calls that do not contain %n and whose return value is not used -- this is not true, and even if it were, no implementation would do so. It's sad that the comment got 25 upvotes, but that's SO for you. – Cupulate

@JimBalter No. That sentence solely means "theoretically it's not forbidden", but I also said "I am unaware if anyone is doing that", – Gibbie

And I'm saying that even "theoretically" it isn't true. I wasn't commenting on your answer, but rather EricPostpichii's comment. And "No" to what, exactly? – Cupulate

console output is pretty slow so the time saved by writting to /dev/null could be significant -- this isn't relevant. The question is about the costs of printf when writing to /dev/null vs. not calling printf at all ... the cost of writing to the console doesn't enter into it. – Cupulate

Writing to a terminal isn't necessarily slow -- people are using the word "slow" as if it meant something, and making completely irrelevant references to terminals, displays, consoles, and files. The question is whether redirecting stdout to /dev/null is as fast as eliding the printfs altogether. The clear answer is "no". – Cupulate

Answer 2

The printf function will write to stdout. It is not conforming to optimize for /dev/null. Therefore, you will have the overhead of parsing the format string and evaluating any necessary arguments, and you will have at least one syscall, plus you will copy a buffer to kernel address space (which, compared to the cost of the syscall is neglegible).

This answer is based on the specific documentation of POSIX.

System Interfaces
dprintf, fprintf, printf, snprintf, sprintf - print formatted output

The fprintf() function shall place output on the named output stream. The printf() function shall place output on the standard output stream stdout. The sprintf() function shall place output followed by the null byte, '\0', in consecutive bytes starting at *s; it is the user's responsibility to ensure that enough space is available.

Base Definitions
shall
For an implementation that conforms to POSIX.1-2017, describes a feature or behavior that is mandatory. An application can rely on the existence of the feature or behavior.

Laylalayman

The kernel is responsible for copying the buffer to kernel address space, and the null driver probably omits that step (on Linux, it definitely does - it does not even check that it is a valid address) – Alkylation

Maybe make it a bit clearer, that not every call to printf will result in a syscall - at the moment this is a bit ambiguous, otherwise great answer. – Markhor

-1, this ignores the as-if rule (which applies to the entire implementation, not just the compiler). If the implementation as a whole can prove that there is no difference in observable behavior, it can make any optimization it likes, even after compilation has completed. Now, if we were talking about a language other than C, you might be right. – Whalen

@Whalen Does the as-if rule of the C standard apply to the POSIX standard too? – Mavis

@Whalen Since calls to I/O functions are observable behavior themselves, isn't optimization restricted. If the standard said that only I/O is observable effect, then removing printf to /dev/null would be possible. – Grubb

@Agnew: printf is listed in the POSIX standard as "confirming to the C standard," so yes it does. – Whalen

@Kevin: No, it says "aligned with", that's different. On the contrary: Language-Dependent Services for the C Programming Language states: "Implementors seeking to claim conformance using the ISO C standard shall claim POSIX conformance as described in POSIX Conformance". – Laylalayman

@jinawee: No, they are not. Observable behavior consists of 1. accesses of volatile variables, 2. data written to files, 3. data written to "interactive devices" (ttys), and 4. the FPU mode. Data "written" to /dev/null is none of those four things. – Whalen

@Whalen Unless there has been some modification wrt. C++ 2003, that's not observable behavior: The observable behavior of the abstract machine is its sequence of reads and writes to volatile data and calls to library I/O functions. In fact, "interactive devices" is only mentioned in the requirement The input and output dynamics of interactive devices shall take place in such a fashion that prompting messages actually appear prior to a program waiting for input. What constitutes an interactive device is implementation-defined. – Grubb

@Whalen -1, this ignores the as-if rule A bit late, but the as-if rule can not be reliably applied here - and certainly not for an entire long-lived program - because whatever stdout happens to be connected to can change dynamically. And it is perfectly conforming with all versions of the C standard to not perform any eliding under the as-if rule, so POSIX requiring all calls to printf() to result in actual writes to stdout in no way violates the C standard. – Brisling

Answer 3

The printf function writes to stdout. If the file descriptor connected to stdout is redirected to /dev/null then no output will be written anywhere (but it will still be written), but the call to printf itself and the formatting it does will still happen.

Extracellular

@OP: Addition: You can further reduce the cost of printf() by creating a new driver which provides a new FILE * (depending on if your platform supports that). In this case, you can create a data sink which discards the data. The cost for formatting etc. still remains, but the OS call for writing to /dev/null goes away. – Sammie

Answers like this should state they are based on general understanding of common implementations and not upon specific documentation. In theory, there is no reason a C implementation might not inspect stdout, learn it is /dev/null, and suppress printf calls that do not contain %n and whose return value is not used. We cannot really assert nobody has done this, and students ought to learn the provenance of information since an important part of engineering is knowing how you know something (is it specified in a standard, is it just assumed, is it provable, and so on). – Sung

Answer 4

Write your own that wraps printf() using the printf() source as a guideline, and returning immediately if a noprint flag is set. The downside of this is when actually printing it will consume more resources because of having to parse the format string twice. But it uses negligible resources when not printing. Can't simply replace printf() because the underlying calls inside printf() can change with a newer version of the stdio library.

void printf2(const char *formatstring, ...);

Skiles

If you use vprintf inside printf2 you will not have to parse the format string twice. – Earlie

Answer 5

Generally speaking, an implementation is permitted to perform such optimisations if they do not affect the observable (functional) outputs of the program. In the case of printf(), that would mean that if the program doesn't use the return value, and if there are no %n conversions, then the implementation would be allowed to do nothing.

In practice, I'm not aware of any implementation on Linux that currently (early 2019) performs such an optimisation - the compilers and libraries I'm familiar with will format the output and write the result to the null device, relying on the kernel' to ignore it.

You may want to write a forwarding function of your own if you really need to save the cost of formatting when the output is not used - you'll want to it to return void, and you should check the format string for %n. (You could use snprintf with a NULL and 0 buffer if you need those side-effects, but the savings are unlikely to repay the effort invested).

Flor

Since the observable behavior of the abstract machine includes calls to library I/O functions, surely printf at least it would have to be called right? Although I'm not sure how two different calls to a library function are considered different or not... – Grubb

Answer 6

in C writing 0; does execute and nothing, which is similar to ;.

means you can write a macro like

#if DEBUG
#define devlognix(frmt,...) fprintf(stderr,(frmt).UTF8String,##__VA_ARGS__)
#else
#define nadanix 0
#define devlognix(frmt,...) nadanix
#endif
#define XYZKitLogError(frmt, ...) devlognix(frmt)

where XYZKitLogError would be your Log command. or even

#define nadanix ;

which will kick out all log calls at compile time and replace with 0; or ; so it gets parsed out.

you will get Unused variable warnings, but it does what you want and this side effect can even be helpful because it tells you about computation that is not needed in your release.

.UTF8String is an Objective-C method converting NSStrings to const char* - you don't need.

Professional