What to do with "The input line is too long" error message?

Question

I am trying to use os.system() to call another program that takes an input and an output file. The command I use is ~250 characters due to the long folder names.

When I try to call the command, I'm getting an error: The input line is too long.

I'm guessing there's a 255 character limit (its built using a C system call, but I couldn't find the limitations on that either).

I tried changing the directory with os.chdir() to reduce the folder trail lengths, but when I try using os.system() with "..\folder\filename" it apparently can't handle relative path names. Is there any way to get around this limit or get it to recognize relative paths?

Answer 1

Assuming you're using windows, from the backslashes, you could write a .bat file from python and then os.system() on that. It's a hack.

Answer 2

Make sure when you're using '\' in your strings that they're being properly escaped.

Python uses the '\' as the escape character, so the string "..\folder\filename" evaluates to "..folderfilename" since an escaped f is still an f.

You probably want to use

r"..\folder\filename"

or

"..\\folder\\filename"

Answer 3

You should use the subprocess module instead. See this little doc for how to rewrite os.system calls to use subprocess.

Answer 4

You should use subprocess instead of os.system.

subprocess has the advantage of being able to change the directory for you:

import subprocess
my_cwd = r"..\folder\"
my_process = subprocess.Popen(["command name", "option 1", "option 2"], cwd=my_cwd)
my_process.wait() # wait for process to end
if my_process.returncode != 0:
    print "Something went wrong!"

The subprocess module contains some helper functions as well if the above looks a bit verbose.

Answer 5

Even it's a good idea to use subprocess.Popen(), this does not solve the issue.

Your problem is not the 255 characters limit, this was true on DOS times, later increased to 2048 for Windows NT/2000, and increased again to 8192 for Windows XP+.

The real solution is to workaround a very old bug in Windows APIs: _popen() and _wpopen().

If you ever use quotes during the command line you have to add the entire command in quoates or you will get the The input line is too long error message.

All Microsoft operating systems starting with Windows XP had a 8192 characters limit which is now enough for any decent command line usage but they forgot to solve this bug.

To overcome their bug just include your entire command in double quotes, and if you want to know more real the MSDN comment on _popen().

Be careful because these works:

prog
"prog"
""prog" param"
""prog" "param""

But these will not work:

""prog param""

If you need a function that does add the quotes when they are needed you can take the one from http://github.com/ssbarnea/tendo/blob/master/tendo/tee.py