Is it possible to temporarily redirect stdout/stderr in Python (i.e. for the duration of a method)?
The problem with the current solutions (which I at first remembered but then forgot) is that they don't redirect; rather, they just replace the streams in their entirety. Hence, if a method has a local copy of one the variable for any reason (e.g. because the stream was passed as a parameter to something), it won't work.
Any solutions?
To solve the issue that some function might have cached sys.stdout stream as a local variable and therefore replacing the global sys.stdout won't work inside that function, you could redirect at a file descriptor level (sys.stdout.fileno()) e.g.:
from __future__ import print_function
import os
import sys
def some_function_with_cached_sys_stdout(stdout=sys.stdout):
print('cached stdout', file=stdout)
with stdout_redirected(to=os.devnull), merged_stderr_stdout():
print('stdout goes to devnull')
some_function_with_cached_sys_stdout()
print('stderr also goes to stdout that goes to devnull', file=sys.stderr)
print('stdout is back')
some_function_with_cached_sys_stdout()
print('stderr is back', file=sys.stderr)
stdout_redirected() redirects all output for sys.stdout.fileno() to a given filename, file object, or file descriptor (os.devnull in the example).
stdout_redirected() and merged_stderr_stdout() are defined here.
You can also put the redirection logic in a contextmanager.
import os
import sys
class RedirectStdStreams(object):
def __init__(self, stdout=None, stderr=None):
self._stdout = stdout or sys.stdout
self._stderr = stderr or sys.stderr
def __enter__(self):
self.old_stdout, self.old_stderr = sys.stdout, sys.stderr
self.old_stdout.flush(); self.old_stderr.flush()
sys.stdout, sys.stderr = self._stdout, self._stderr
def __exit__(self, exc_type, exc_value, traceback):
self._stdout.flush(); self._stderr.flush()
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
if __name__ == '__main__':
devnull = open(os.devnull, 'w')
print('Fubar')
with RedirectStdStreams(stdout=devnull, stderr=devnull):
print("You'll never see me")
print("I'm back!")
sys.stdout, sys.stderr instead of redirecting them as OP asked in the edited version of the question. See my answer that does redirect them. –
Lichee self._stdout "protected" (in the Python sense), vs self.__stdout "private" (in the Python sense)? –
Photomechanical x._bar and x.__bar except that x.__bar is harder to get at, debugging wise due to name-mangling. It doesn't actually make the variable private, so it doesn't buy you anything, but makes your debugging life more difficult. In general, stick with a single underscore unless you're dealing with multiple inheritance and have a particular reason to want dunderscore (both classes' variables need to exist? I feel like there are more elegant solutions) –
Giaimo with is better as with open(os.devnull, "w") as devnull, RedirectStdStreams(stdout=devnull, stderr=devnull): so that devnull is also closed fine –
Disown starting from python 3.4 there is the context manager contextlib.redirect_stdout:
from contextlib import redirect_stdout
with open('yourfile.txt', 'w') as f:
with redirect_stdout(f):
# do stuff...
to completely silence stdout this works:
from contextlib import redirect_stdout
with redirect_stdout(None):
# do stuff...
redirect_stdout(None) and it didn't work. The module producing the output is written in C++, maybe that has something to do with it. –
Kunz caffe.Net() in with redirect_stderr(None) and still got a load of debug messaged dumped to stderr. Turns out you can disable those for the specific case of Caffe another way but I guess disabling standard output is not that easy for modules that use std::iostream under the hood. –
Kunz with redirect_stdout(sys.stderr): and then you could redirect stderr (or the other way round). –
Peppergrass sys.stdout directly you'll need to redirect it to /dev/null, rather than setting it to None, like so: with redirect_stdout(open(os.devnull, "w")) –
Topographer To solve the issue that some function might have cached sys.stdout stream as a local variable and therefore replacing the global sys.stdout won't work inside that function, you could redirect at a file descriptor level (sys.stdout.fileno()) e.g.:
from __future__ import print_function
import os
import sys
def some_function_with_cached_sys_stdout(stdout=sys.stdout):
print('cached stdout', file=stdout)
with stdout_redirected(to=os.devnull), merged_stderr_stdout():
print('stdout goes to devnull')
some_function_with_cached_sys_stdout()
print('stderr also goes to stdout that goes to devnull', file=sys.stderr)
print('stdout is back')
some_function_with_cached_sys_stdout()
print('stderr is back', file=sys.stderr)
stdout_redirected() redirects all output for sys.stdout.fileno() to a given filename, file object, or file descriptor (os.devnull in the example).
stdout_redirected() and merged_stderr_stdout() are defined here.
I am not sure what temporary redirection means. But, you can reassign streams like this and reset it back.
temp = sys.stdout
sys.stdout = sys.stderr
sys.stderr = temp
Also to write to sys.stderr within print stmts like this.
print >> sys.stderr, "Error in atexit._run_exitfuncs:"
Regular print will to stdout.
try-finally around that; seems like it'd work, though not as pretty as I wanted. Thanks +1 –
Claudie temp variable. You can call sys.stdout, sys.stderr = sys.stderr, sys.stdout. –
Apiarist It's possible with a decorator such as the following:
import sys
def redirect_stderr_stdout(stderr=sys.stderr, stdout=sys.stdout):
def wrap(f):
def newf(*args, **kwargs):
old_stderr, old_stdout = sys.stderr, sys.stdout
sys.stderr = stderr
sys.stdout = stdout
try:
return f(*args, **kwargs)
finally:
sys.stderr, sys.stdout = old_stderr, old_stdout
return newf
return wrap
Use as:
@redirect_stderr_stdout(some_logging_stream, the_console):
def fun(...):
# whatever
or, if you don't want to modify the source for fun, call it directly as
redirect_stderr_stdout(some_logging_stream, the_console)(fun)
But note that this is not thread-safe.
except: raise? –
Claudie except: raise thing, it was a thinko on my part. I rewrote the whole thing btw.; I always get the level of nesting in decorators wrong the first time, but it actually works now. –
Brandybrandyn Here's a context manager that I found useful. The nice things about this are that you can use it with the with statement and it also handles redirecting for child processes.
import contextlib
@contextlib.contextmanager
def stdchannel_redirected(stdchannel, dest_filename):
"""
A context manager to temporarily redirect stdout or stderr
e.g.:
with stdchannel_redirected(sys.stderr, os.devnull):
...
"""
try:
oldstdchannel = os.dup(stdchannel.fileno())
dest_file = open(dest_filename, 'w')
os.dup2(dest_file.fileno(), stdchannel.fileno())
yield
finally:
if oldstdchannel is not None:
os.dup2(oldstdchannel, stdchannel.fileno())
if dest_file is not None:
dest_file.close()
The context for why I created this is at this blog post.
Raymond Hettinger shows us a better way[1]:
import sys
with open(filepath + filename, "w") as f: #replace filepath & filename
with f as sys.stdout:
print("print this to file") #will be written to filename & -path
After the with block the sys.stdout will be reset
[1]: http://www.youtube.com/watch?v=OSGv2VnC0go&list=PLQZM27HgcgT-6D0w6arhnGdSHDcSmQ8r3
with will not restore the initial sys.stdout. If you try to print something after those with blocks you receive a ValueError: I/O operation on closed file.. You should remove the second with and put a sys.stdout = sys.__stdout__ at the end of the first with block. –
Joule Look at contextlib.redirect_stdout(new_target) and contextlib.redirect_stderr(new_target). redirect_stderr is new in Python 3.5.
We'll use the PHP syntax of ob_start and ob_get_contents functions in python3, and redirect the input into a file.
The outputs are being stored in a file, any type of stream could be used as well.
from functools import partial
output_buffer = None
print_orig = print
def ob_start(fname="print.txt"):
global print
global output_buffer
print = partial(print_orig, file=output_buffer)
output_buffer = open(fname, 'w')
def ob_end():
global output_buffer
close(output_buffer)
print = print_orig
def ob_get_contents(fname="print.txt"):
return open(fname, 'r').read()
Usage:
print ("Hi John")
ob_start()
print ("Hi John")
ob_end()
print (ob_get_contents().replace("Hi", "Bye"))
Would print
Hi John Bye John
© 2022 - 2024 — McMap. All rights reserved.
stderrorstdoutwired-in, you can't always avoid using that kind of code. – Brandybrandyn.write()). All references to sys.stdout point to it. Have it proxy to a changeable stream, defaulting to stdout. You should then have the ability to switch the proxied stream at will. I haven't tried this; I'm thinking out loud. – Gambesonsys.stdoutandsys.stderr. There issys.__stdout__andsys.__stderr__. (reference) – Maniac