What to do with "The input line is too long" error message?

Question

I am trying to use os.system() to call another program that takes an input and an output file. The command I use is ~250 characters due to the long folder names.

When I try to call the command, I'm getting an error: The input line is too long.

I'm guessing there's a 255 character limit (its built using a C system call, but I couldn't find the limitations on that either).

I tried changing the directory with os.chdir() to reduce the folder trail lengths, but when I try using os.system() with "..\folder\filename" it apparently can't handle relative path names. Is there any way to get around this limit or get it to recognize relative paths?

Grackle

Answer 1

You should use the subprocess module instead. See this little doc for how to rewrite os.system calls to use subprocess.

Inappetence

WARNING: this is not a solution, please check next answer and you will discover that there is a bug in windows API, triggered when you use double quotes in command lines. – Alidaalidade

Answer 2

Even it's a good idea to use subprocess.Popen(), this does not solve the issue.

Your problem is not the 255 characters limit, this was true on DOS times, later increased to 2048 for Windows NT/2000, and increased again to 8192 for Windows XP+.

The real solution is to workaround a very old bug in Windows APIs: _popen() and _wpopen().

If you ever use quotes during the command line you have to add the entire command in quoates or you will get the The input line is too long error message.

All Microsoft operating systems starting with Windows XP had a 8192 characters limit which is now enough for any decent command line usage but they forgot to solve this bug.

To overcome their bug just include your entire command in double quotes, and if you want to know more real the MSDN comment on _popen().

Be careful because these works:

prog
"prog"
""prog" param"
""prog" "param""

But these will not work:

""prog param""

If you need a function that does add the quotes when they are needed you can take the one from http://github.com/ssbarnea/tendo/blob/master/tendo/tee.py

Alidaalidade

@Sorbin - This was a huge help. I used it on Windows7 without any of the stuff you were talking about above. One thing I did have to do was add an extra double quote to the start. i.e. """prog" "param"". Not yet sure why that is so, I have my hunches and will prove /disprove those now. – Herisau

His problem was "The command line is too long" not "The input line is too long" – Rehnberg

Answer 3

You should use subprocess instead of os.system.

subprocess has the advantage of being able to change the directory for you:

import subprocess
my_cwd = r"..\folder\"
my_process = subprocess.Popen(["command name", "option 1", "option 2"], cwd=my_cwd)
my_process.wait() # wait for process to end
if my_process.returncode != 0:
    print "Something went wrong!"

The subprocess module contains some helper functions as well if the above looks a bit verbose.

Romeu

Answer 4

Assuming you're using windows, from the backslashes, you could write a .bat file from python and then os.system() on that. It's a hack.

Hartung

.bat file does not accept commands longer than 8192 characters either. – Welcher

Answer 5

Make sure when you're using '\' in your strings that they're being properly escaped.

Python uses the '\' as the escape character, so the string "..\folder\filename" evaluates to "..folderfilename" since an escaped f is still an f.

You probably want to use

r"..\folder\filename"

or

"..\\folder\\filename"

Abrupt

yeah, I'm using raw strings, good catch for me not mentioning that. – Grackle

Answer 6

I got the same message but it was strange because the command was not that long (130 characters) and it used to work, it just stopped working one day. I just closed the command window and opened a new one and it worked. I have had the command window opened for a long time (maybe months, it's a remote virtual machine). I guess is some windows bug with a buffer or something.

Mildredmildrid

It is because invoking set mypath=%mypath%;appendix makes your path appendix longer every time you call it. Ultimately, guess what, it becomes too long. This where your windows leaks a bug. This question elaborates your problem, stackoverflow.com/questions/19630441 – Trestle