How to escape os.system() calls in Python?

Question

When using os.system() it's often necessary to escape filenames and other arguments passed as parameters to commands. How can I do this? Preferably something that would work on multiple operating systems/shells but in particular for bash.

I'm currently doing the following, but am sure there must be a library function for this, or at least a more elegant/robust/efficient option:

def sh_escape(s):
   return s.replace("(","\\(").replace(")","\\)").replace(" ","\\ ")

os.system("cat %s | grep something | sort > %s" 
          % (sh_escape(in_filename), 
             sh_escape(out_filename)))

Edit: I've accepted the simple answer of using quotes, don't know why I didn't think of that; I guess because I came from Windows where ' and " behave a little differently.

Regarding security, I understand the concern, but, in this case, I'm interested in a quick and easy solution which os.system() provides, and the source of the strings is either not user-generated or at least entered by a trusted user (me).

Answer 1

Beware of the security issue! For instance if out_filename is

foo.txt; rm -rf /

The malicious user can add more command directly interpreted by the shell.

Answer 2

I believe that os.system just invokes whatever command shell is configured for the user, so I don't think you can do it in a platform independent way. My command shell could be anything from bash, emacs, ruby, or even quake3. Some of these programs aren't expecting the kind of arguments you are passing to them and even if they did there is no guarantee they do their escaping the same way.

Answer 3

This is what I use:

def shellquote(s):
    return "'" + s.replace("'", "'\\''") + "'"

The shell will always accept a quoted filename and remove the surrounding quotes before passing it to the program in question. Notably, this avoids problems with filenames that contain spaces or any other kind of nasty shell metacharacter.

Answer 4

Perhaps you have a specific reason for using os.system(). But if not you should probably be using the subprocess module. You can specify the pipes directly and avoid using the shell.

The following is from PEP324

Replacing shell pipe line
-------------------------

output=`dmesg | grep hda`
==>
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

Answer 5

If you do use the system command, I would try and whitelist what goes into the os.system() call.. For example..

clean_user_input re.sub("[^a-zA-Z]", "", user_input)
os.system("ls %s" % (clean_user_input))

The subprocess module is a better option, and I would recommend trying to avoid using anything like os.system/subprocess wherever possible.

Answer 6

pipes.quote() (available since python 1.6) does what you want.

Answer 7

pipes.quote() (available since python 1.6) does what you want. ~~~pixelbeat

thanks, pixelbeat!

>>> from pipes import quote
>>> args=['a a',r'b<\">B', '''c"''C''']
>>> print ' '.join( quote( arg ) for arg in args )
'a a' 'b<\">B' "c\"''C"

Answer 8

Note that pipes.quote is actually broken and not safe to use--It doesn't handle zero-length arguments.

>>> from pipes import quote
>>> args = ['arg1', '', 'arg3']
>>> print 'mycommand %s' % (' '.join(quote(arg) for arg in args))
mycommand arg1  arg3

Answer 9

The function I use is:

def quote_argument(argument):
    return '"%s"' % (
        argument
        .replace('\\', '\\\\')
        .replace('"', '\"')
        .replace('$', '\$')
        .replace('`', '\`')
    )

that is: I always enclose the argument in double quotes, and then backslash-quote the only characters special inside double quotes.