How to get data from command line from within a Python program?

Question

I want to run a command line program from within a python script and get the output.

How do I get the information that is displayed by foo so that I can use it in my script?

For example, I call foo file1 from the command line and it prints out

Size: 3KB
Name: file1.txt
Other stuff: blah

How can I get the file name doing something like filename = os.system('foo file1')?

Piotr

possible duplicate of Executing command line programs from within python – Adiana

Answer 1

Use the subprocess module:

import subprocess

command = ['ls', '-l']
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.IGNORE)
text = p.stdout.read()
retcode = p.wait()

Then you can do whatever you want with variable text: regular expression, splitting, etc.

The 2nd and 3rd parameters of subprocess.Popen are optional and can be removed.

Aficionado

... where command should be ['foo', 'file1', 'file2', ...]. +1. – Levey

Thanks! So the foo output is all in text... Now how would I get the file name out of that? Is it like an array or something where I can get text[1]? Or can I search through it for "Name:"? Or start at line #2 character #7 thru newline? – Piotr

Look at the re module and write a simple regular expression that pulls it in pat = re.compile(r'Name:\s+(.*)$') m = pat.search(text) if m: m.group(1) (you'll need to add some newlines) – Aficionado

I just realized that I have to use Python v2.4, which doens't let me do stdout=subprocess.PIPE. How else can one go about doing this? – Piotr

subprocess and PIPE should be there in 2.4 - docs.python.org/release/2.4/lib/node227.html the stderr to ignore is missing. – Aficionado

Ah, figured it out. This is a problem only on Windows. Not sure what the right way to fix it is, but I found some workarounds. – Piotr

'module' object has no attribute 'IGNORE' – Lelia

It is better to use stderr=subprocess.PIPE, then stdout_data, stderr_data = p.communicate(), then stdout_data, stderr_data = stdout_data.decode('utf-8'), stderr_data.decode('utf-8') – Gilletta

Answer 2

The easiest way to get the output of a tool called through your Python script is to use the subprocess module in the standard library. Have a look at subprocess.check_output.

>>> subprocess.check_output("echo \"foo\"", shell=True)
'foo\n'

(If your tool gets input from untrusted sources, make sure not to use the shell=True argument.)

Ordain

@PrakashPandey this has been answered here: #23421490 – Amsterdam

Answer 3

This is typically a subject for a bash script that you can run in python :

#!/bin/bash
# vim:ts=4:sw=4

for arg; do
    size=$(du -sh "$arg" | awk '{print $1}')
    date=$(stat -c "%y" "$arg")
    cat<<EOF
Size: $size
Name: ${arg##*/}
Date: $date 
EOF

done

Edit : How to use it : open a pseuso-terminal, then copy-paste this :

cd
wget http://pastie.org/pastes/2900209/download -O info-files.bash

In python2.4 :

import os
import sys

myvar = ("/bin/bash ~/info-files.bash '{}'").format(sys.argv[1])
myoutput = os.system(myvar) # myoutput variable contains the whole output from the shell
print myoutput

Seto

I know nothing about bash. How do I use this with Python? – Piotr

This solution is for Linux only or Windows too if you have Cygwin en.wikipedia.org/wiki/Cygwin If you are using windows or if you want a portable solution, see my new post – Good

Answer 4

refer to https://docs.python.org/3/library/subprocess.html#subprocess.run

from subprocess import run
o = run("python q2.py",capture_output=True,text=True)
print(o.stdout)

the output will be encoded

from subprocess import run
o = run("python q2.py",capture_output=True)
print(o.stdout)

base syntax

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, text=None, env=None, universal_newlines=None, **other_popen_kwargs)

here output will not be encoded just a tip

Amyotonia

Answer 5

This is a portable solution in pure python :

import os
import stat
import time

# pick a file you have ...
file_name = 'll.py'
file_stats = os.stat(file_name)

# create a dictionary to hold file info
file_info = {
    'fname': file_name,
    'fsize': file_stats [stat.ST_SIZE],
    'f_lm': time.strftime("%m/%d/%Y %I:%M:%S %p",time.localtime(file_stats[stat.ST_MTIME])),
}


print("""
Size: {} bytes
Name: {}
Time: {}
 """
).format(file_info['fsize'], file_info['fname'], file_info['f_lm'])

Seto

Answer 6

In Python, You can pass a plain OS command with spaces, subquotes and newlines into the subcommand module so we can parse the response text like this:

1. Save this into test.py:

   #!/usr/bin/python
   import subprocess
   
   command = ('echo "this echo command' + 
   ' has subquotes, spaces,\n\n" && echo "and newlines!"')
   
   p = subprocess.Popen(command, universal_newlines=True, 
   shell=True, stdout=subprocess.PIPE, 
   stderr=subprocess.PIPE)
   text = p.stdout.read()
   retcode = p.wait()
   print text;
   

2. Then run it like this:

   python test.py 
   

3. Which prints:

   this echo command has subquotes, spaces,
   
   
   and newlines!
   

If this isn't working for you, it could be troubles with the python version or the operating system. I'm using Python 2.7.3 on Ubuntu 12.10 for this example.

Straightaway