Python 3 writing to a pipe

Question

I'm trying to write some code to put data into a pipe, and I'd like the solution to be python 2.6+ and 3.x compatible. Example:

from __future__ import print_function

import subprocess
import sys

if(sys.version_info > (3,0)):
    print ("using python3")
    def raw_input(*prmpt):
        """in python3, input behaves like raw_input in python2"""
        return input(*prmpt)

class pipe(object):
    def __init__(self,openstr):
        self.gnuProcess=subprocess.Popen(openstr.split(),
                                         stdin=subprocess.PIPE)

    def putInPipe(self,mystr):
        print(mystr, file=self.gnuProcess.stdin)

if(__name__=="__main__"):
    print("This simple program just echoes what you say (control-d to exit)")
    p=pipe("cat -")
    while(True):
        try:
            inpt=raw_input()
        except EOFError:
            break
        print('putting in pipe:%s'%inpt)
        p.putInPipe(inpt)

The above code works on python 2.6 but fails in python 3.2 (Note that the above code was mostly generated with 2to3 -- I just messed with it a little to make it python 2.6 compatible.)

Traceback (most recent call last):
  File "test.py", line 30, in <module>
   p.putInPipe(inpt)
  File "test.py", line 18, in putInPipe
   print(mystr, file=self.gnuProcess.stdin)
TypeError: 'str' does not support the buffer interface

I've tried the bytes function (e.g. print(bytes(mystr,'ascii')) suggested here, TypeError: 'str' does not support the buffer interface But that doesn't seem to work. Any suggestions?

Cahill

Trying to comply Python 2 and 3 at the same time is very hard and unneeded. Just write idioamtic, modern (not relying on stuff that's removed completely in 3.x) code and 2to3 should work. If there are a few places where it doesn't, keep a patch changing it to be 3.x-compatible and apply it to 2to3's output. – Sabellian

I understand that the point of 2to3 is to prevent the user from needing to support python2 and python3 in the same file. However, I did that above to illustrate the problem. 2to3 converted print >> self.gnuProcess.stdin,mystr to print(mystr,file=self.gnuProcess.stdin). However, the converted output does not work (raises TypeError). how do I write the code so that (at very least) 2to3 will convert it to something that is functional? – Cahill

@delan: Better yet, write python3 code and translate it back with the 3to2 tool. – Stringent

@ThomasAhle -- Not sure that I agree with this. if you target python 3, then you're likely to use some super cool feature/module/function which doesn't exist in python2. 3to2 won't really help you with that one... – Cahill

Answer 1

The print function converts its arguments to a string representation, and outputs this string representation to the given file. The string representation always is of type str for both, Python 2.x and Python 3.x. In Python 3.x, a pipe only accepts bytes or buffer objects, so this won't work. (Even if you pass a bytes object to print, it will be converted to a str.)

A solution is to use the write() method instead (and flushing after writing):

self.gnuProcess.stdin.write(bytes(mystr + "\n", "ascii"))
self.gnuProcess.stdin.flush()

Sauveur

Answer 2

but python2 will complain about

bytes("something", "ascii")

if you use the mutable bytearray it will work in both python2 and python3 unaltered

self.gnuProcess.stdin.write(bytearray(mystr + "\n", "ascii"))
self.gnuProcess.stdin.flush()

Saveloy