command line arg parsing through introspection

Question

I'm developing a management script that does a fairly large amount of work via a plethora of command-line options. The first few iterations of the script have used optparse to collect user input and then just run down the page, testing the value of each option in the appropriate order, and doing the action if necessary. This has resulted in a jungle of code that's really hard to read and maintain.

I'm looking for something better.

My hope is to have a system where I can write functions in more or less normal python fashion, and then when the script is run, have options (and help text) generated from my functions, parsed, and executed in the appropriate order. Additionally, I'd REALLY like to be able to build django-style sub-command interfaces, where myscript.py install works completely separately from myscript.py remove (separate options, help, etc.)

I've found simon willison's optfunc and it does a lot of this, but seems to just miss the mark — I want to write each OPTION as a function, rather than try to compress the whole option set into a huge string of options.

I imagine an architecture involving a set of classes for major functions, and each defined method of the class corresponding to a particular option in the command line. This structure provides the advantage of having each option reside near the functional code it modifies, easing maintenance. The thing I don't know quite how to deal with is the ordering of the commands, since the ordering of class methods is not deterministic.

Before I go reinventing the wheel: Are there any other existing bits of code that behave similarly? Other things that would be easy to modify? Asking the question has clarified my own thinking on what would be nice, but feedback on why this is a terrible idea, or how it should work would be welcome.

Answer 1

Don't waste time on "introspection".

Each "Command" or "Option" is an object with two sets of method functions or attributes.

1. Provide setup information to optparse.

2. Actually do the work.

Here's the superclass for all commands

class Command( object ):
    name= "name"
    def setup_opts( self, parser ):
        """Add any options to the parser that this command needs."""
        pass
    def execute( self, context, options, args ):
        """Execute the command in some application context with some options and args."""
        raise NotImplemented

You create sublcasses for Install and Remove and every other command you need.

Your overall application looks something like this.

commands = [ 
    Install(),
    Remove(),
]
def main():
    parser= optparse.OptionParser()
    for c in commands:
        c.setup_opts( parser )
    options, args = parser.parse()
    command= None
    for c in commands:
        if c.name.startswith(args[0].lower()):
            command= c
            break
    if command:
        status= command.execute( context, options, args[1:] )
    else:
        logger.error( "Command %r is unknown", args[0] )
        status= 2
    sys.exit( status )

Answer 2

The WSGI library werkzeug provides Management Script Utilities which may do what you want, or at least give you a hint how to do the introspection yourself.

from werkzeug import script

# actions go here
def action_test():
    "sample with no args"
    pass

def action_foo(name=2, value="test"):
    "do some foo"
    pass

if __name__ == '__main__':
    script.run()

Which will generate the following help message:

$ python /tmp/test.py --help
usage: test.py <action> [<options>]
       test.py --help

actions:
  foo:
    do some foo

    --name                        integer   2
    --value                       string    test

  test:
    sample with no args

An action is a function in the same module starting with "action_" which takes a number of arguments where every argument has a default. The type of the default value specifies the type of the argument.

Arguments can then be passed by position or using --name=value from the shell.