Where does my embedded python stdout go?

Question

Consider the following MWE:

#include <Python.h>
#include <stdio.h>

int main(void) {
  printf("Test 1\n");
  Py_Initialize();
  printf("Test 2\n");
  PyRun_SimpleString("print('Test 3')");
  printf("Test 4\n");
  return 0;
}

When I compile and run this as normal i get the expected output:

$ ./test
Test 1
Test 2
Test 3
Test 4

But when I redirect the output I get nothing from the python code:

$ ./test | cat
Test 1
Test 2
Test 4

What is happening? And more importantly how do I get my python output written to stdout like expected?

Bra

Interesting that if you add 'import sys; sys.stdout.flush()' to the Python script, you get the redirected output, but before everything else. – Brave

Answer 1

When stdout refers to a terminal, the output is line buffered otherwise which is block or fully buffered, won't output until the block is full.

To make the output line buffered when stdout refers to non-terminal, set the mode with setvbuf
And you have to call Py_Finalize() to have libpython close its I/O handle.

#include <Python.h>
#include <stdio.h>

int
main(int argc, char **argv) {
    //setlinebuf(stdout);
    setvbuf(stdout, NULL, _IOLBF, 0);
    printf("Test 1\n");
    Py_Initialize();
    printf("Test 2\n");
    PyRun_SimpleString("print('Test 3')");
    Py_Finalize();
    printf("Test 4\n");
    return 0;
}

Prospectus

You can call sys.stdout.flush() in Python. Calling Py_Finalize() might not be intenional for his use case. – Chiba

@Chiba if you don't call Py_Finalize() the output is out of order, that's Python's is first. – Prospectus

Ah... of course, it all seams so obvious now :) Thank you! – Bra

Answer 2

In current Python, you can use Py_UnbufferedStdioFlag.

Laxative