Suppressing output of module calling outside library

Question

I have an annoying problem when using machine learning library PyML. PyML uses libsvm to train the SVM classifier. The problem is that libsvm outputs some text to standard output. But because that is outside of Python I cannot intercept it. I tried using methods described in problem Silence the stdout of a function in Python without trashing sys.stdout and restoring each function call but none of those help.

Is there any way how to do this. Modifying PyML is not an option.

Berget

have you check it , maybe it write in sys.stderr and not sys.stdout !!! – Fruitage

related: Redirect stdout to a file in Python? -- the file is os.devnull in this case. – Liken

Answer 1

Open /dev/null for writing, use os.dup() to copy stdout, and use os.dup2() to copy your open /dev/null to stdout. Use os.dup2() to copy your copied stdout back to the real stdout after.

devnull = open('/dev/null', 'w')
oldstdout_fno = os.dup(sys.stdout.fileno())
os.dup2(devnull.fileno(), 1)
makesomenoise()
os.dup2(oldstdout_fno, 1)

Brezin

'oldstdout = os.dup(sys.stdout)' throws a typeerror 'an integer is required' – Berget

figured it out, sys.stdout.fileno() and devnull.fileno() are required and it works after that, thanks!! – Berget

Shouldn't you close devnull at the end using devnull.close() or open it using the with statement? – Father

Reid, your answer still does not work for me. I needed a os.fsync(devnull.fileno()) before the final dup2 To debug, try import pybullet as pb as the thing you want to silence :) – Delldella

os.dup2(0, 1) on systems that don't have /dev/null :) (os.dup2(sys.stdin.fileno(), sys.stdout.fileno()) more generally.) – Paleozoic

Answer 2

Dave Smith gave a wonderful answer to that on his blog. Basically, it wraps Ignacio's answer nicely:

def suppress_stdout():
    with open(os.devnull, "w") as devnull:
        old_stdout = sys.stdout
        sys.stdout = devnull
        try:  
            yield
        finally:
            sys.stdout = old_stdout

Now, you can surround any function that garbles unwanted noise into stdout like this:

print "You can see this"
with suppress_stdout():
    print "You cannot see this"
print "And you can see this again"

For Python 3 you can use:

from contextlib import contextmanager
import os
import sys

@contextmanager
def suppress_stdout():
    with open(os.devnull, "w") as devnull:
        old_stdout = sys.stdout
        sys.stdout = devnull
        try:  
            yield
        finally:
            sys.stdout = old_stdout

Embryogeny

These solutions can only suppress output produced within python. – Israelite

Probably, yeah. Also, I found that this solution fails to suppress some python output, for example the TQDM progress bar manages to print to Stdout even when within this context, not sure why. Still, I found that for most cases in Python, it works. – Embryogeny

TQDM progressbar is printed to stderr by default to supress it you would also need to redirect sys.stderr – Bugleweed

@Bugleweed that doesn't seem to work for me in a ipynb – Naumann

The ipynb renders progressbar as html if I'm not mistaken. That means it cannot be suppressed by these means. – Bugleweed

Answer 3

I had a similar problem with portaudio/PyAudio initialization. I started with Reid's answer, which worked. Although I needed to redirect stderr instead. So, here is an updated, cross-platform version that redirects both:

import sys, os

# hide diagnostic output
with open(os.devnull, 'w') as devnull:
    # suppress stdout
    orig_stdout_fno = os.dup(sys.stdout.fileno())
    os.dup2(devnull.fileno(), 1)
    # suppress stderr
    orig_stderr_fno = os.dup(sys.stderr.fileno())
    os.dup2(devnull.fileno(), 2)

    print('*** stdout should be hidden!  ****')
    print('*** stderr should be too!  ****',
          file=sys.stderr)

    os.dup2(orig_stdout_fno, 1)  # restore stdout
    os.dup2(orig_stderr_fno, 2)  # restore stderr

print('done.')

Should be easy to comment out a part you don't need.

Edit: These might help, don't have time to look at the moment:

https://docs.python.org/3/library/contextlib.html#contextlib.redirect_stdout

Buddleia

This doesn't work on Windows, stdout and stderr do not get restored (the last print doesn't reach stdout). – Israelite

Answer 4

I had the same problem and fixed it like that:

from cStringIO import StringIO

def wrapped_svm_predict(*args):
    """Run :func:`svm_predict` with no *stdout* output."""
    so, sys.stdout = sys.stdout, StringIO()
    ret = svm_predict(*args)
    sys.stdout = so
    return ret

Chilt

This will work if svm_predict is the output comes from python but not if it comes from a shared library that python calls, which I believe is what the OP was asking about. – Giustino

For me it worked using the Python bindings of libsvm. I guess for PyML the situation is similar. Stdout is a process property, i.e. my wrapping (as well as Ignacio's solution) perfectly works as long as external functionality is not used via subprocesses. It only would fail if someone opens /dev/stdout to write output to. – Chilt

Which is exactly what most shared libraries do, which is why this solution will not work for most people. The above solution is going to be closer to what the requestor needs. – Pastoralize