Why doesn't print output show up immediately in the terminal when there is no newline at the end?

Question

I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:

for step in steps:
    run_simulation(step)
    # Python 3.x version:
    print('.', end='')
    # for Python 2.x:
    # print '.',

However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?

---

_{This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.}

Ultrared

Line buffering, most likely. Try issuing a sys.stdout.flush() after you print. If that solves your issue, buffering is actually your problem. – Pustulant

You could also set stdout to unbuffered… but the usual way to do something like this is exactly what @LukasGraf suggests. Except you probably want sys.stdout.write('.'), not print '.',. First, print will give you spaces between the dots. Second, it's theoretically allowed to do its own buffering on top of whatever stdout does (although CPython doesn't, and I don't know of any implementation that does). – Latecomer

Yes it worked. Thanks!! I wonder if there's a better to do it than just printing the dots though... – Ultrared

sys.stdout.write is what I recommend as well. For a workaround (e.g. you can't modify the source code), you can try running Python with -u as described here – Monocular

@Ultrared I don't know about better. But you could use a curses-based Text UI framework like Urwid and draw a fancy progress bar. Urwid would then automatically take care of buffering / screen redrawing issues for you. But it's a lot more work and would add a significant dependency to your project. – Pustulant

Even though your question is about why, the obvious follow up question would be "How do I turn off buffering then?", so I voted to mark it as a duplicate. – Pustulant

I have reopened this, and am promoting it as a canonical. Many people run into this issue, and most will not know about the issue of buffering, and thus not phrase searches in those terms. This doesn't simply duplicate that material, because some intervening explanation is necessary. Since there is already a good version of the question, and an authoritative CW answer, it makes more sense to direct those beginner questions here, and then include the appropriate reference links. – Arondell

Answer 1

The issue

By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.

Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.

So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.

Outputting in other ways

Every now and then, someone will try to output from a Python program by using the standard output stream directly:

import sys
sys.stdout.write('test')

This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.

Fixing the issue

For a single print

We can explicitly flush the output after printing.

In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:

for _ in range(10):
    print('.', end=' ', flush=True)
    time.sleep(.2)  # or other time-consuming work

In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:

for _ in range(10):
    print '.',
    sys.stdout.flush()
    time.sleep(.2)  # or other time-consuming work

For multiple prints

Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.

Arondell