When python3 chain two subprocess.run (such as bash pipe) get error "AttributeError: 'bytes' object has no attribute 'fileno'"

Question

The bash code is:

mx=8;my=8;head -c "$((3*mx*my))" /dev/urandom | convert -depth 8 -size "${mx}x${my}" RGB:- /tmp/random.png

And the code in python3:

#!/usr/bin/python3

import subprocess

result = subprocess.run(['head', '-c {}'.format(3 * size * size), '/dev/urandom'], stdout=subprocess.PIPE)
print('{}'.format(result))
result = subprocess.run(['convert', '-depth 8', '-size{}x{}'.format(size, size), 'RGB:-', '/tmp/random.png'], stdin=result.stdout, stdout=subprocess.PIPE)

But the output is:

$ python3 create_mosaic_images.py 
CompletedProcess(args=['head', '-c 192', '/dev/urandom'], returncode=0, stdout=b"j\xd1U0\x7f\xc0\x11\x9e\xcdJ\x88P\xdc\xe5\xd26\xee\xf3m\xe0\xf9S\xd5%0q\xfb\x01\xd4^\xbd^R\xe4\x9c\x9e\xb5\xaf\x99:B[\xdc4\x80\x1a\x8a>\xeb\x9e\xab\xae/\xa4\xf4\xc3u\xaca\x8efJ\x9fNK\xa2\xb0\x18\x98\x95\xb08\xeb\x07\xf6\x9d2Ko\x11\xa0\x98\xf7\x8b\x04'\xd4;\xe8\xa5SD\xc7\xf8\xe6;\x8b\x880\xb8\x1e~\xee>r\xf0\xc0g\xfb\xce\xd6\xfcq\x1a\x91\xa0t0\x96\xf47\x9cj\xd4\x9ac\x15z\x81\xf35\x8f\x1ay\xa2\x8av\xb5\xe4\x15ox\x00\x00O\xfa\x98\x17\xe1\x04\xd5\x8dx\xdf\x9a.\xce<b\xb2\x16\x15\x94\xbd\x17Z\x14\xba\xc5-?\xbcH\t\x9a~1y\xc5\xe6\xf5\xd2[\xfb\xc0k\x90\xdfB\xafje")
Traceback (most recent call last):
  File "create_mosaic_images.py", line 26, in <module>
    generate_random_pixels_square()
  File "create_mosaic_images.py", line 19, in generate_random_pixels_square
    result = subprocess.run(['convert', '-depth 8', '-size{}x{}'.format(size, size), 'RGB:-', '/tmp/random.png'], stdin=result.stdout, stdout=subprocess.PIPE)
  File "/usr/lib/python3.5/subprocess.py", line 693, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.5/subprocess.py", line 911, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib/python3.5/subprocess.py", line 1392, in _get_handles
    p2cread = stdin.fileno()
AttributeError: 'bytes' object has no attribute 'fileno'

Well, I don't know what is bad.

Scramble

Answer 1

you cannot chain 2 run calls like this.

You need the first call to use Popen, where result.stdout is a real file (result.stdout returned by run is a bytes object, not a real file, because when run returns, the process is already ended, with all output produced, as opposed as Popen which starts the process and writes to a pipe)

process1 = subprocess.Popen(['head', '-c','{}'.format(3 * size * size), '/dev/urandom'], stdout=subprocess.PIPE)
result = subprocess.Popen(['convert', '-depth','8', '-size{}x{}'.format(size, size), 'RGB:-', '/tmp/random.png'], stdin=process1.stdout, stdout=subprocess.PIPE)

Aside, your argument splitting was wrong. You cannot include spaces, you have to perform the full split.

A better way in that case would be to zap the first process, since you can do that in python already:

with open('/dev/urandom','rb') as f:
    random_block = b"".join(f.readline() for _ in range(3 * size * size))
    result = subprocess.Popen(['convert', '-depth','8', '-size{}x{}'.format(size, size), 'RGB:-', '/tmp/random.png'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    output,error = result.communicate(random_block);

Bertilla

Unfortunately, the official docs recommend using subprocess.run "for all cases it can handle" without doing a very good of explaining what those are, this looks like one of them – Digit

Answer 2

In the second subprocess run, you need to use input= instead of stdin=

result = subprocess.run(['convert', '-depth 8', '-size{}x{}'.format(size, size), 'RGB:-', '/tmp/random.png'], input=result.stdout, stdout=subprocess.PIPE)

Ricoriki

Answer 3

Let shlex split the shell command for you.

import subprocess
import shlex

cmd = shlex.split('''convert -depth 8 -size{}x{} RGB:- /tmp/random.png'''.format(size, size))
result = subprocess.run(cmd, stdin=result.stdout, stdout=subprocess.PIPE)

Vanitavanity