Does new line character also flush the buffer?

Question

I understand that questions like, difference between endl and \n have been answered many times on SO. But they only mention that endl is able to flush the buffer onto the stdout, while \n, does not.

So, what I understand by buffer being flushed is that, the input given is stored in a buffer, and is passed onto the stdout only when it comes across endl, or some explict flush functions. If so, I expected that the following code :

#include <iostream>
#include <unistd.h>

int main(void)
{
    std::cout << "Hello\nworld";
    sleep(2);
    std::cout << std::endl;

    return 0;
}

To display:

After 2 seconds

Hello
World

But the actual output was:

Hello

After 2 seconds

World

Why is it so ?

shouldn't \n also be stored in the buffer and only when endl is encountered buffer is to be flushed/displayed onto the stdout, but from what I observe \n is acting the same way as endl.

Frampton

It depends on where cout is going. If it goes to a terminal ('interactive device'), then it can't be fully buffered — it is usually line buffered, meaning that characters appear after a newline is printed, or could in theory be unbuffered. If it is going to a pipe or file or other non-interactive destination, the endl forces the data out even if the stream is fully buffered, as it usually will be. – Billyebilobate

I also wanted to know if I provided neither new line character nor endl , will the output be displayed on the stdout once it reaches the end of the program, I know it does for terminal, but is it applicable to all types of stdout? – Frampton

Yes, when the file stream is closed at the (normal) end of the program, pending output will be flushed. It'll also be flushed when the buffer is full. If the program aborts, pending output usually won't be flushed. – Billyebilobate

@JonathanLeffler: the C++ way to deal with output to interactive streams is to have std::cin be tie()d to std::cout: whenever std::cin is accessed, std::cout is flushed. The actual reason line buffer behaviour is observed for C++ programs writing to stdout is the synchronization with stdio which is, obviously, disabled for any sane C++ program using standard streams as the program is otherwise unreasonably slowed down. – Mucosa

Answer 1

^{Converting comments into an answer.}

It depends on where cout is going. If it goes to a terminal ('interactive device'), then it can't be fully buffered — it is usually line buffered, meaning that characters appear after a newline is printed, or could in theory be unbuffered. If it is going to a pipe or file or other non-interactive destination, the endl forces the data out even if the stream is fully buffered, as it usually will be.

I also wanted to know if I provided neither new line character nor endl, will the output be displayed on the stdout once it reaches the end of the program, I know it does for terminal, but is it applicable to all types of stdout?

Yes, when the file stream is closed at the (normal) end of the program, pending output will be flushed. It'll also be flushed when the buffer is full. If the program aborts, pending output usually won't be flushed.

Billyebilobate

Answer 2

The default setup for the standard C++ stream objects (std::cin, std::cout, std::cerr, and std::clog) is that they are synchronized with the corresponding C streams (stdin, stdout, and stderr). Synchronization means that alternating access of the C++ and the C streams results in consistent behaviour. For example, this code is expected to produce the string hello, world:

std::cout << "hel";
fprintf(stdout, "lo,");
std::cout << " wo";
fprintf(stdout, "rld");

The C++ standard makes no mandate on how this synchronization is implemented. One way to implement it is to disable any buffering for std::cout (and family) and immediately access stdout. That is, the above example could immediately write the individual characters to stdout.

If the characters are actually written to stdout the default setting for the buffering mode for stdout would be used. I can't find a specification in the standard but typically the default for the buffering mode of stdout is _IOLBF when it is connected to an interactive stream (e.g., a console), i.e., the buffer is flushed at the end of lines. The default for writing to a file is typically _IOFBF, i.e., the output is flushed when a complete buffer is written. As a result, writing a newline to std::cout could result in the buffer being flushed.

The streams in C++ are normally set up to be buffered. That is, writing a newline to a file will generally not cause the output to appear immediately (it would only appear immediately if the character caused the buffer to overflow the stream is set to be unbuffered). Since the synchronization with stdout is often unnecessary, e.g., when a program always uses std::cout to write to the standard output, but does cause output to the standard output to be slowed down quite dramatically (disabling buffering for stream makes them slow) the synchronization can be disabled:

std::ios_base::sync_with_stdio(false);

This disables synchronization for all stream objects. For a bad implementation there could be no effect while a good implementation will enable buffering for std::cout resulting in a substantial speed-up and probably also disabling line buffering.

Once a C++ stream is buffered, there is no built-in way to cause it to be flushed when a newline is written. The primary reason for this is that dealing with line buffering would require inspection of each character by the stream buffer which effectively inhibits bulk operations on characters and thereby causing a substantial slow-down. If needed, line buffering can be implemented through a simple filtering stream buffer. For example:

class linebuf: public std::streambuf {
    std::streambuf* sbuf;
public:
    linebuf(std::streambuf* sbuf): sbuf(sbuf) {}
    int_type overflow(int_type c) {
        int rc = this->sbuf->sputc(c);
        this->sbuf->pubsync();
        return rc;
    }
    int sync() { return this->sbuf->pubsync(); }
};
// ...
int main() {
    std::ios_base::sync_with_stdio(false);
    linebuf sbuf(std::cout.rdbuf());
    std::streambuf* origcout = std::cout.rdbuf(&sbuf);

    std::cout << "line\nbuffered\n";

    std::cout.rdbuf(origcout); // needed for clean-up;
}

tl;dr: the C++ standard doesn't have a concept of line buffering but it may get it when standard I/O is synchronized from C's behaviour of stdout.

Domash