Python subprocess, realtime print with COLORS and save stdout

Question

Printing the output of a subprocess while saving the result is not a new problem, and has been answered many times before e.g.: https://mcmap.net/q/25515/-constantly-print-subprocess-output-while-process-is-running This does not work for me because I am trying to maintain the shell colours printed. E.g. when one goes systemctl status application, its prints running in green. The above-mentioned methods all rely on reading a line one by one from subprocess, but it seems to me by then the colour information is stripped off and lost.

I tried to make an object which tee's off the stdout prints and saves them into a variable:

from subprocess import *
import sys

class Tee():
    def __init__(self):
        self.content = ''
        self.stdout = sys.stdout
        sys.stdout = self
    def __enter__(self):
        return self
    def __exit__(self, *args):
        pass
    def __del__(self):
        sys.stdout = self.stdout
    def write(self, data):
        self.content += data
        self.stdout.write(data)
    def flush(self):
        self.content = ''

with Tee() as tee:
    # Saves print to tee.content
    print("Hello World")

    # This line does not save prints to tee.content    
    run(['apt-get', 'update'])

    # raises an error that tee.fileno is not supported
    run(['systemctl', 'status', 'nginx'], stdout=tee)

    content = tee.content

print("---------------------")
print(content)

But the problem is subprocess's stdout requires an actual file: https://mcmap.net/q/1158601/-creating-a-custom-sys-stdout-class

Is there anyway to print realtime the output of a subprocess, while maintaining the colours, and store the value to a variable (without going through a temp file)?

Twopiece

Answer 1

You can't make it with subprocess, but pty can. pty creates pseudo-terminal so the command being executed detects that it's running with tty and enables color output.

import pty, os

output_bytes = []

def read(fd):
    data = os.read(fd, 1024)
    output_bytes.append(data)
    return data

pty.spawn([command], read)
output = str(output_bytes)
# parse output as you need

Sevenup

This will not work in windows because tty termios does not exist in windows. – Moria

Answer 2

Execute command in list form with live colorised output:

cmd_parts = ['mycommand', 'arg1', 'arg2']

import os, pty
def exec_live_output(cmd_parts):
    pty.spawn(cmd_parts, lambda fd: os.read(fd, 1024))

Papistry

Answer 3

systemctl checks where the output is going; if it's a tty, it shows colorized output. If the STDOUT is not attached to a tty, it does not show the color.

So, essentially you need to do this from source i.e. make systemctl emit necessary escape codes when the STDOUT is not a tty.

There is a way, from man systemd:

$SYSTEMD_COLORS
    Controls whether colorized output should be generated.

So you need to pass the SYSTEMD_COLORS environment variable to make systmectl return output with color escapes.

You can do:

os.environ['SYSTEMD_COLORS'] = '1'
subprocess.run(['systemctl', 'status', 'application'], stdout=subprocess.PIPE)

To have the env var for this command only do del os.environ['SYSTEMD_COLORS'] afterwards.

Or run on shell directly for single command only:

subprocess.run('SYSTEMD_COLORS=1 systemctl status application', shell=True, stdout=subprocess.PIPE)

Handling

This answer is disappointing since it solves the problem just for systemctl colors. This is one case where actually posting the exact problem produced an answer that is helpful just for the original poster and does not provide a general method for similar situations. The solution is more about system administration, not programming. – Labors