How to use psutil.Popen with unicode commands on Python 2

Question

Hey I'm trying to execute the following command (using psutil.Popen with python 2.7):

"C:\Program Files (x86)\Microsoft Office\root\Office16\EXCEL.EXE" "C:\docs\ת.xlsm"

Using this code:

dir = u"C:\\docs"
doc = os.listdir(dir)[0]
full_path = os.path.join(dir, doc)
command = u"\"C:\\Program Files (x86)\\Microsoft Office\\root\\Office16\\EXCEL.EXE\" \"{}\"".format(full_path)
process = psutil.Popen(command)

But I'm getting this exception:

    process = psutil.Popen(command)
  File "C:\Python27\lib\site-packages\psutil\__init__.py", line 1370, in __init__
    self.__subproc = subprocess.Popen(*args, **kwargs)
  File "C:\Python27\lib\subprocess.py", line 390, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 640, in _execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u05ea' in position 102: ordinal not in range(128)

I have found this related question: subprocess.Popen with a unicode path. But every given answer there doesn't work for me.

Using subprocess.Popen(command.encode(locale.getpreferredencoding())), throws the following exception:

Traceback (most recent call last):
  File "scanner_watcher.py", line 53, in _execute
    self.scanner.execute(path)
  File "scanner_watcher.py", line 356, in execute
    self._execute(file_path)
  File "scanner_watcher.py", line 201, in _execute
    self.open_scanner(file_path, file_package)
  File "scanner_watcher.py", line 287, in open_scanner
    self.scanner_process = psutil.Popen(command.encode(locale.getpreferredencoding()))
  File "C:\Python27\lib\encodings\cp1252.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u05ea' in position 102: character maps to <undefined>

Using path.encode('mbcs') turns all the Unicode characters into question marks.

I don't want to use os.startfile because I need to use different commands on the program and then handle the opened process (when the os.startfile doesn't allow that).

I've found this modified Popen: https://gist.github.com/vaab/2ad7051fc193167f15f85ef573e54eb9 But this code was not thoroughly tested.

Is there a correct way to use Popen with a Unicode command in python 2.7?

Thanks.

Renn

Please show code, so we can see exactly what format this command is in and exactly what you're doing to attempt to execute it. – Speech

@Speech Added the code – Renn

Your options are to use ctypes, cffi, or an extension module to call CreateProcessW with a Unicode command line. – Urn

You could also use the C runtime's _wsystem function. For example: import ctypes; msvcr90 = ctypes.CDLL('msvcr90', use_errno=True); wsystem = msvcr90._wsystem; wsystem.argtypes = (ctypes.c_wchar_p,). Then call wsystem(u'"%s"' % command). The extra double quotes get stripped off by CMD's "old behavior", for which you can read the rules via cmd /?. – Urn

@eryksun That's interesting.. But I want to use Popen to eventually use the pipe options, to get the output for example, etc.. – Renn

I'm just looking for a simpler call than CreateProcessW. You could try _wpopen, but working with C FILE streams via ctypes starts to approach the complexity of using CreateProcessW. – Urn

Answer 1

The issue is your use of double-quotes when defining command. The way it's written splits the statement into multiple strings in all the wrong places. Substituting single-quotes for the first and final quotes (open and close of string) fixed the issue for me.

command = u'\"C:\\Program Files (x86)\\Microsoft Office\\root\\Office16\\EXCEL.EXE\" \"{}\"'.format(full_path)

Outnumber