Python optparse and spaces in an argument

Question

When using optparse i want to get the whole string after an option, but I only get part of it up to the first space.

e.g.:

python myprog.py --executable python someOtherProg.py

What I get in 'executable' is just 'python'.

Is it possible to parse such lines using optparse or do you have to use argparse to do it?

€: I have already tried enclosing it in "s. But after digging further into the code I found out that the subprocess invocation can't handle the argument.

The string with the commandline gets crammed into a list 'args'.

args = [self.getExecutable()] + self.getArgs().split()

It's like

"[python D:\\\workspace\\\myprog\\\src\\\myprog.py]"

That gives me the System can't find file exception. When I use

args[0]

it works. But I loose the arguments to the executable.

The subprocess module builds a cmdline from a list if it does not get a string in the first place, so I can't explain that behavior at the moment.

Answer 1

You can enclose them in quotes to make them work with the existing code.

python myprog.py --executable "python someOtherProg.py"

Is it possible to parse such lines using optparse or do you have to use argparse to do it?

I don't know if/how you can do it with optparse as I haven't really worked with optparse.

I can however help you out with argparse. Here is a quick example:

#!/usr/bin/python
import argparse, sys

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description = 'Demonstration of Argparse.')
    parser.add_argument('-e', '--executable', nargs = '+', help = 'List of executables')
    args = parser.parse_args(sys.argv[1:])
    print args.executable

And usage:

manoj@maruti:~$ python myprog.py --executable python someOtherProg.py
['python', 'someOtherProg.py']

I'd also recommend switching from optparse to argparse. Optparse is deprecated since 2.7.

Answer 2

The behavior you see comes from the fact that it's your shell, not python, that parses the command line and separates it into the words of sys.argv. Python is launched by the shell via exec() with argv already populated.

Most shells will split argv items at spaces unless you tell them not to by quoting or escaping.

Quotes work as described above.

In many shells you could do this:

python myprog.py --executable python\ someOtherProg.py

The backslash escapes the following space without requiring quotes.

Answer 3

If you know how many words after the argument flag you are going to get, you can modify the way you create the --executable option in in optparse to properly handle the situation:

Instead of taking a single word after the option flag you can set the optparse parser to look for two (or more) words:

from optparse import OptionParser
parser = OptionParser()

parser.add_option("-f", "--file", action="store", dest="filename",
                       help="File to be processed.", metavar="FILE")
parser.add_option("-e", "--executable", action="store", dest="my_exe",
                       help="Command to be executed", metavar="EXE",
                       nargs=2)

In this snippet, the -f or --file option only expects a single word and stores it as a string (the default) in the filename variable.

In contrast the -e, --executable option expects two words because of the nargs=2 option. This will result in the two words found behind the -e or --executable flag to be stored as strings in a Python list my_exe.

Check out: http://docs.python.org/library/optparse.html for more info on optparse, and remember it has been deprecated as of 2.7 in favour of argparse.