Using **kwargs with SimpleXMLRPCServer in python

Question

I have a class that I wish to expose as a remote service using pythons SimpleXMLRPCServer. The server startup looks like this:

server = SimpleXMLRPCServer((serverSettings.LISTEN_IP,serverSettings.LISTEN_PORT))

service = Service()

server.register_instance(service)
server.serve_forever()

I then have a ServiceRemote class that looks like this:

def __init__(self,ip,port):
    self.rpcClient = xmlrpclib.Server('http://%s:%d' %(ip,port))

def __getattr__(self, name):
    # forward all calls to the rpc client
    return getattr(self.rpcClient, name)

So all calls on the ServiceRemote object will be forwarded to xmlrpclib.Server, which then forwards it to the remote server. The problem is a method in the service that takes named varargs:

@useDb
def select(self, db, fields, **kwargs):
    pass

The @useDb decorator wraps the function, creating the db before the call and opening it, then closing it after the call is done before returning the result.

When I call this method, I get the error "call() got an unexpected keyword argument 'name'". So, is it possible to call methods taking variable named arguments remotely? Or will I have to create an override for each method variation I need.

Answer 1

As far as I know, the underlying protocol doesn't support named varargs (or any named args for that matter). The workaround for this is to create a wrapper that will take the **kwargs and pass it as an ordinary dictionary to the method you want to call. Something like this

Server side:

def select_wrapper(self, db, fields, kwargs):
    """accepts an ordinary dict which can pass through xmlrpc"""
    return select(self,db,fields, **kwargs)

On the client side:

def select(self, db, fields, **kwargs):
    """you can call it with keyword arguments and they will be packed into a dict"""
    return self.rpcClient.select_wrapper(self,db,fields,kwargs)

Disclaimer: the code shows the general idea, you can do it a bit cleaner (for example writing a decorator to do that).

Answer 2

XML-RPC doesn't really have a concept of 'keyword arguments', so xmlrpclib doesn't try to support them. You would need to pick a convention, then modify xmlrpclib._Method to accept keyword arguments and pass them along using that convention.

For instance, I used to work with an XML-RPC server that passed keyword arguments as two arguments, '-KEYWORD' followed by the actual argument, in a flat list. I no longer have access to the code I wrote to access that XML-RPC server from Python, but it was fairly simple, along the lines of:

import xmlrpclib

_orig_Method = xmlrpclib._Method

class KeywordArgMethod(_orig_Method):     
    def __call__(self, *args, **kwargs):
        if args and kwargs:
            raise TypeError, "Can't pass both positional and keyword args"
        args = list(args) 
        for key in kwargs:
            args.append('-%s' % key.upper())
            args.append(kwargs[key])
       return _orig_Method.__call__(self, *args)     

xmlrpclib._Method = KeywordArgMethod

It uses monkeypatching because that's by far the easiest method to do this, because of some clunky uses of module globals and name-mangled attributes (__request, for instance) in the ServerProxy class.

Answer 3

As Thomas Wouters said, XML-RPC does not have keyword arguments. Only the order of arguments matters as far as the protocol is concerned and they can be called anything in XML: arg0, arg1, arg2 is perfectly fine, as is cheese, candy and bacon for the same arguments.

Perhaps you should simply rethink your use of the protocol? Using something like document/literal SOAP would be much better than a workaround such as the ones presented in other answers here. Of course, this may not be feasible.

Answer 4

Thanks for the responses. I changed my code around a bit so the question is no longer an issue. However now I know this for future reference if I indeed do need to implement positional arguments and support remote invocation. I think a combination of Thomas and praptaks approaches would be good. Turning kwargs into positional args on the client through xmlrpclient, and having a wrapper on methods serverside to unpack positional arguments.

Answer 5

Using the above advice, I created some working code.

Server method wrapper:

def unwrap_kwargs(func):
    def wrapper(*args, **kwargs):
        print args
        if args and isinstance(args[-1], list) and len(args[-1]) == 2 and "kwargs" == args[-1][0]:
            func(*args[:-1], **args[-1][1])
        else:
            func(*args, **kwargs)
    return wrapper

Client setup (do once):

_orig_Method = xmlrpclib._Method

class KeywordArgMethod(_orig_Method):     
    def __call__(self, *args, **kwargs):
        args = list(args) 
        if kwargs:
            args.append(("kwargs", kwargs))
        return _orig_Method.__call__(self, *args)

xmlrpclib._Method = KeywordArgMethod

I tested this, and it supports method with fixed, positional and keyword arguments.

Answer 6

You can't do this with plain xmlrpc since it has no notion of keyword arguments. However, you can superimpose this as a protocol on top of xmlrpc that would always pass a list as first argument, and a dictionary as a second, and then provide the proper support code so this becomes transparent for your usage, example below:

Server

from SimpleXMLRPCServer import SimpleXMLRPCServer

class Server(object):
    def __init__(self, hostport):
        self.server = SimpleXMLRPCServer(hostport)

    def register_function(self, function, name=None):
        def _function(args, kwargs):
            return function(*args, **kwargs)
        _function.__name__ = function.__name__
        self.server.register_function(_function, name)

    def serve_forever(self):
        self.server.serve_forever()

#example usage
server = Server(('localhost', 8000))
def test(arg1, arg2):
    print 'arg1: %s arg2: %s' % (arg1, arg2)
    return 0
server.register_function(test)
server.serve_forever()

Client

import xmlrpclib

class ServerProxy(object):
    def __init__(self, url):
        self._xmlrpc_server_proxy = xmlrpclib.ServerProxy(url)
    def __getattr__(self, name):
        call_proxy = getattr(self._xmlrpc_server_proxy, name)
        def _call(*args, **kwargs):
            return call_proxy(args, kwargs)
        return _call

#example usage
server = ServerProxy('http://localhost:8000')
server.test(1, 2)
server.test(arg2=2, arg1=1)
server.test(1, arg2=2)
server.test(*[1,2])
server.test(**{'arg1':1, 'arg2':2})