How to run " ps cax | grep something " in Python?

Question

How do I run a command with a pipe | in it?

The subprocess module seems complex...

Is there something like

output,error = `ps cax | grep something`

as in shell script?

Nikos

Don't do this with subprocess. It's much easier to do this with the shell. Indeed, this is the one thing the shell does best. – Appertain

Answer 1

See Replacing shell pipeline:

import subprocess

proc1 = subprocess.Popen(['ps', 'cax'], stdout=subprocess.PIPE)
proc2 = subprocess.Popen(['grep', 'python'], stdin=proc1.stdout,
                         stdout=subprocess.PIPE, stderr=subprocess.PIPE)

proc1.stdout.close() # Allow proc1 to receive a SIGPIPE if proc2 exits.
out, err = proc2.communicate()
print('out: {0}'.format(out))
print('err: {0}'.format(err))

PS. Using shell=True can be dangerous. See for example the warning in the docs.

---

There is also the sh module which can make subprocess scripting in Python a lot more pleasant:

import sh
print(sh.grep(sh.ps("cax"), 'something'))

Tangleberry

Great reference to the sh module, I highly recommend it. – Afflict

sh.grep(sh.ps('aux', _piped=True), 'something') - worked for me – Splanchnology

@unutbu, Could you explain more detail why proc1.stdout.close() should be called? – Averil

@SangminKim: This appears to have been fixed or worked-around in Python3, but in Python2.7 at least, if you were to spawn a long-running process such as find / -print in proc1 without calling proc1.stdout.close() and spawn a short-running command like head in proc2, then you would see find / -print is still running long after the call to head has ended. Including proc1.stdout.close() allows find to end soon after head finishes. – Tangleberry

For more on SIGPIPE, see this comment, this post and this one too. – Tangleberry

@unutbu, Thank you for your answering. However, I am not clear on that. Thus I open another question on here, since It is too long to write as a comment. – Averil

ImportError: No module named sh – Blastula

Answer 2

You've already accepted an answer, but:

Do you really need to use grep? I'd write something like:

import subprocess
ps = subprocess.Popen(('ps', 'cax'), stdout=subprocess.PIPE)
output = ps.communicate()[0]
for line in output.split('\n'):
    if 'something' in line:
        ...

This has the advantages of not involving shell=True and its riskiness, doesn't fork off a separate grep process, and looks an awful lot like the kind of Python you'd write to process data file-like objects.

Herve

for me, output was a [byte] and had to use it like str(output).split("\\n") – Leshalesher

Answer 3

import subprocess

process = subprocess.Popen("ps cax | grep something",
                             shell=True,
                             stdout=subprocess.PIPE,
                           )
stdout_list = process.communicate()[0].split('\n')

Personality

something in grep something would be a variable, should I construct the "ps cax | grep something" string with variable to run it? or is there other way of doing it? – Nikos

docs.python.org/library/stdtypes.html#string-formatting or docs.python.org/library/string.html#string-formatting. Basically, 'grep %s % variable`. – Apure

@Eugene: You can construct the string using a variable, but be careful about where the variable is coming from. I.e. make sure it's not from a user who could make "something" into "something; rm -rf /". Building expressions to run with shell=True is a possible security risk. – Hugely

Answer 4

Drop that 'ps' subprocess and back away slowly! :)

Use the psutil module instead.

Quotation

I want to use psutil, but this answer isn't very helpful without showing how it would be done. – Energid

pls share example – Cupped

import psutil; for proc in psutil.process_iter(): cmdline = " ".join(proc.cmdline()); if something in cmdline: break – Nozzle

Answer 5

import os

os.system('ps -cax|grep something')

If you wanna replace grep argument with some variable:

os.system('ps -cax|grep '+your_var)

Boloney

@Kirk Strauser, can't you get the output? I have testd it both in interactive python interpreter and running it as script, it works as expected(my python version is 2.7.1,if it helps) – Boloney

os.system only returns the int exit code of the subprocess. If you're running it at an interactive prompt and seeing the output of ps, it's because ps is writing to stout. Python isn't actually capturing that output. Try it yourself: run a = os.system('ls'). You'll still see the output of ls, and a will be 0 (assuming ls didn't fail for some reason). – Herve

This is unsafe. If your_var is set by someone else, this code lets them run any command they want. For example your_var = "a; echo you got hacked" – Schilling