Redirect stdout from python for C calls

Question

This is a follow up question from here specifically concerning its answer.

---

From a python module I am calling a Hello World executable that simply prints Hello World to the stdout. I am interested in redirecting that output to a python StringIO and ran into this answer which almost brings me all the way to the solution.

The critical part of this answer is this code segment:

1. def redirect_stdout():
2.     print "Redirecting stdout"
3.     sys.stdout.flush() # <--- important when redirecting to files
4.     newstdout = os.dup(1)
5.     devnull = os.open('/dev/null', os.O_WRONLY)
6.     os.dup2(devnull, 1)
7.     os.close(devnull)
8.     sys.stdout = os.fdopen(newstdout, 'w')

Also I would like to restore the stdout as it was before the redirection.

Questions

1. What exactly is going on in the function above?
   • What is dup and dup2 doing?
   • What is /dev/null?
   • What is line 8 doing? (sys.stdout = os.fdopen(newstdout, 'w'))
2. How can I store the stdout in a StringIO object?
3. How can I restore the stdout after the call to my Hello World program?

I am pretty sure that once I have the answer for my question 1 that the answers of questions 2 and 3 will be easy. I decided to post them anyway to maybe push the answer of question 1 into the direction where I want to go.

Copaiba

Answer 1

I've written below a few additional comments that should make clearer what it's going on inside the redirect_stdout function:

def redirect_stdout():
    print "Redirecting stdout"
    sys.stdout.flush() # <--- important when redirecting to files

    # Duplicate stdout (file descriptor 1)
    # to a different file descriptor number
    newstdout = os.dup(1)

    # /dev/null is used just to discard what is being printed
    devnull = os.open('/dev/null', os.O_WRONLY)

    # Duplicate the file descriptor for /dev/null
    # and overwrite the value for stdout (file descriptor 1)
    os.dup2(devnull, 1)

    # Close devnull after duplication (no longer needed)
    os.close(devnull)

    # Use the original stdout to still be able
    # to print to stdout within python
    sys.stdout = os.fdopen(newstdout, 'w')

One important thing to note is that a process gets three different file descriptors from the OS when launched:

• stdin: 0
• stdout: 1
• stderr: 2

As explained in the comments, the code above takes advantage of the file descriptor for stdout and the file descriptor duplication functions to make trick the C code into using a different stdout while still keeping a reference to the original stdout in the python code to be able to print.

Burhans

If I have redirected the stdout with redirect_stdout. How would I restore the stdout again so that external programs can print to stdout? os.dup2(1, devnull) does not seem to be working?! – Copaiba

It should be os.dup2(newstdout, 1) – Burhans

Answer 2

/dev/null is a special device file (everything in UNIX is a file!) that swallows everything written to it like a black hole. dup duplicates a file descriptor. If you are used to Windows, a file descriptor in UNIX is a special integer which represents an open file, it's like a Windows file handle.

The program is opening /dev/null for writing (only), taking a copy of it's file descriptor, closing the open file (because having a file descriptor is enough for UNIX to write to a file, you don't need to keep the resources open), then assigning the open file to sys.stdout.

Remember sys is the Python module which represents all sorts of system-specific resources, such as the file system. So, in UNIX sys.stdout will represent /dev/stdout i.e. the system STDOUT stream.

So, altogether, this code is fooling Python into thinking that /dev/null/ is STDOUT so now every time your program writes to STDOUT with, say, the print statement (function in Python3) then it will really be writing to /dev/null and you will never see the resulting text on your console.

Arnett

Answer 3

See manual pages for the C runtime functions that underlie these Python functions.

Basically, they duplicate a file descriptor, into either a new file descriptor (with dup()) or into a file descriptor specified in the call, with dup2().

Trivet