subprocess.Popen() IO redirect

Question

Trying to redirect a subprocess' output to a file.

server.py:

while 1:
    print "Count " + str(count)
    sys.stdout.flush()
    count = count + 1
    time.sleep(1)

Laucher:

cmd = './server.py >temp.txt'
args = shlex.split(cmd)
server = subprocess.Popen( args )

The output appear on screen, temp.txt remains empty. What am I doing wrong?

As background I am trying to capture the output of a program that has already been written.

I cannot use:

server = subprocess.Popen(
                [exe_name],
                stdin=subprocess.PIPE, stdout=subprocess.PIPE)

as the program may not flush. Instead I was going to redirect output through a fifo. This works fine if I manually launch server.py but obviously not if I Popen() cause redirect doesnt work. ps -aux shows that server.py was launched correctly.

Bogart

Possible duplicate of How to redirect output with subprocess in Python? – Undershorts

Answer 1

Altenatively, you can use the stdout parameter with a file object:

with open('temp.txt', 'w') as output:
    server = subprocess.Popen('./server.py', stdout=output)
    server.communicate()

As explained in the documentation:

stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are PIPE, an existing file descriptor (a positive integer), an existing file object, and None.

Obsidian

That works. Communicate would block reader until the server terminates but now that I can redirect I can send server output to a fifo. – Bogart

no need to call .communicate() here. subprocess.check_call('command', stdout=file) works. – George

Answer 2

Output redirection with ">" is a feature of shells - by default, subprocess.Popen doesn't instantiate one. This should work:

server = subprocess.Popen(args, shell=True)

Swainson

Using shell=True is not the preferred way, unless it's necessary (and here it is not) – Mentally

@Mentally This seems to be an important comment, but there is no explanation why. – Endometrium