How to capture Python interpreter's and/or CMD.EXE's output from a Python script?

Question

1. Is it possible to capture Python interpreter's output from a Python script?
2. Is it possible to capture Windows CMD's output from a Python script?

If so, which librar(y|ies) should I look into?

Emmaemmalee

Answer 1

If you are talking about the python interpreter or CMD.exe that is the 'parent' of your script then no, it isn't possible. In every POSIX-like system (now you're running Windows, it seems, and that might have some quirk I don't know about, YMMV) each process has three streams, standard input, standard output and standard error. Bu default (when running in a console) these are directed to the console, but redirection is possible using the pipe notation:

python script_a.py | python script_b.py

This ties the standard output stream of script a to the standard input stream of script B. Standard error still goes to the console in this example. See the article on standard streams on Wikipedia.

If you're talking about a child process, you can launch it from python like so (stdin is also an option if you want two way communication):

import subprocess
# Of course you can open things other than python here :)
process = subprocess.Popen(["python", "main.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
x = process.stderr.readline()
y = process.stdout.readline()
process.wait()

See the Python subprocess module for information on managing the process. For communication, the process.stdin and process.stdout pipes are considered standard file objects.

For use with pipes, reading from standard input as lassevk suggested you'd do something like this:

import sys
x = sys.stderr.readline()
y = sys.stdin.readline()

sys.stdin and sys.stdout are standard file objects as noted above, defined in the sys module. You might also want to take a look at the pipes module.

Reading data with readline() as in my example is a pretty naïve way of getting data though. If the output is not line-oriented or indeterministic you probably want to look into polling which unfortunately does not work in windows, but I'm sure there's some alternative out there.

Trash

Note that when invoking Python using subprocess.Popen() it's often helpful to pass the "-u" flag, which disables buffering on stdin/stdout/stderr. Python does not autoflush stdout when the child begins reading stdin, if output has been redirected to a pipe, so you can end up blocking forever reading output that is buffered. I encountered this problem trying to wrap/automate pdb. – Lamere

Answer 2

I think I can point you to a good answer for the first part of your question.

1.  Is it possible to capture Python interpreter's output from a Python script?

The answer is "yes", and personally I like the following lifted from the examples in the PEP 343 -- The "with" Statement document.

from contextlib import contextmanager
import sys

@contextmanager
def stdout_redirected(new_stdout):
    saved_stdout = sys.stdout
    sys.stdout = new_stdout
    try:
        yield None
    finally:
        sys.stdout.close()
        sys.stdout = saved_stdout

And used like this:

with stdout_redirected(open("filename.txt", "w")):
    print "Hello world"

A nice aspect of it is that it can be applied selectively around just a portion of a script's execution, rather than its entire extent, and stays in effect even when unhandled exceptions are raised within its context. If you re-open the file in append-mode after its first use, you can accumulate the results into a single file:

with stdout_redirected(open("filename.txt", "w")):
    print "Hello world"

print "screen only output again"

with stdout_redirected(open("filename.txt", "a")):
    print "Hello world2"

Of course, the above could also be extended to also redirect sys.stderr to the same or another file. Also see this answer to a related question.

Hymeneal

Answer 3

Actually, you definitely can, and it's beautiful, ugly, and crazy at the same time!

You can replace sys.stdout and sys.stderr with StringIO objects that collect the output.

Here's an example, save it as evil.py:

import sys
import StringIO

s = StringIO.StringIO()

sys.stdout = s

print "hey, this isn't going to stdout at all!"
print "where is it ?"

sys.stderr.write('It actually went to a StringIO object, I will show you now:\n')
sys.stderr.write(s.getvalue())

When you run this program, you will see that:

• nothing went to stdout (where print usually prints to)
• the first string that gets written to stderr is the one starting with 'It'
• the next two lines are the ones that were collected in the StringIO object

Replacing sys.stdout/err like this is an application of what's called monkeypatching. Opinions may vary whether or not this is 'supported', and it is definitely an ugly hack, but it has saved my bacon when trying to wrap around external stuff once or twice.

Tested on Linux, not on Windows, but it should work just as well. Let me know if it works on Windows!

Hart

Answer 4

You want subprocess. Look specifically at Popen in 17.1.1 and communicate in 17.1.2.

Ruminate

Answer 5

Is it possible to capture Windows CMD's output from a Python script?

which librar(y|ies) should I look into?

In Windows, you can use the ctypes library and kernel32.dll to grab all of the text that is in the command prompt.

Here is some sample code that I use to copy everything that is written in the command prompt when I run stable diffusion:

#copies the command prompt text to console_text. [comment 1]
import subprocess
import ctypes
CONSOLE_STDOUT_HANDLE = -11
console_handle = ctypes.windll.kernel32.GetStdHandle(CONSOLE_STDOUT_HANDLE)
console_buffer_size = ctypes.windll.kernel32.GetLargestConsoleWindowSize(console_handle)
console_buffer = ctypes.create_string_buffer(console_buffer_size)

console_bytes_read = ctypes.c_ulong(0)
ctypes.windll.kernel32.ReadConsoleOutputCharacterA(console_handle, console_buffer, console_buffer_size, ctypes.c_ulong(0), ctypes.byref(console_bytes_read))
console_text = console_buffer.value[:console_bytes_read.value].decode("windows-1252")


#creates an output file and uses regex to format it nicely. Saves the file as session.txt. [comment 2]
import re
myfile = re.sub(r"\s\s\s\s\s\s+","\n",console_text)
with open("C:\\Users\\name\\Desktop\\stablediff\\_previousSessionTmp\\session.txt", 'w') as f:
print(myfile, file=f)


#prints a statement to let the user know that the information was saved. [comment 3]
command = "echo Session information has been copied to session.txt"
p = subprocess.Popen(command, universal_newlines=True, shell=True, stdout=subprocess.PIPE)
retcode = p.wait()
print(p.stdout.read())

The code below comment 1 is the answer to your question about capturing windows CMD's output. Everything that is currently in the command prompt goes to the console_text variable.

The code below comment 2 saves the text to a file.

The code below comment 3 uses python to echo text into the command prompt, so that if you are looking at the command prompt you will know that all the text before it is saved to the file made in comment 2.

Here's the documentation for ctypes and kernel32.dll

Tool

Answer 6

In which context are you asking?

Are you trying to capture the output from a program you start on the command line?

if so, then this is how to execute it:

somescript.py | your-capture-program-here

and to read the output, just read from standard input.

If, on the other hand, you're executing that script or cmd.exe or similar from within your program, and want to wait until the script/program has finished, and capture all its output, then you need to look at the library calls you use to start that external program, most likely there is a way to ask it to give you some way to read the output and wait for completion.

Vasques