Easy, Robust IPC between Python and PHP

Question

I have a python program which starts up a PHP script using the subprocess.Popen() function. The PHP script needs to communicate back-and-forth with Python, and I am trying to find an easy but robust way to manage the message sending/receiving.

I have already written a working protocol using basic sockets, but it doesn't feel very robust - I don't have any logic to handle dropped messages, and I don't even fully understand how sockets work which leaves me uncertain about what else could go wrong.

Are there any generic libraries or IPC frameworks which are easier than raw sockets?

• ATM I need something which supports Python and PHP, but in the future I may want to be able to use C, Perl and Ruby also.
• I am looking for something robust, i.e. when the server or client crashes, the other party needs to be able to recover gracefully.

Answer 1

You could look at shared memory or named pipes, but I think there are two more likely options, assuming at least one of these languages is being used for a webapp:

A. Use your database's atomicity. In python, begin a transaction, put a message into a table, and end the transaction. From php, begin a transaction, take a message out of the table or mark it "read", and end the transaction. Make your PHP and/or python self-aware enough not to post the same messages twice. Voila; reliable (and scaleable) IPC, using existing web architecture.

B. Make your webserver (assuming as webapp) capable of running both php and python, locking down any internal processes to just localhost access, and then call them using xmlrpc or soap from your other language using standard libraries. This is also scalable, as you can change your URLs and security lock-downs later.

Answer 2

It sounds like you want a generic RPC framework.

You should take a look at:

• Thrift http://incubator.apache.org/thrift/
• XML RPC http://docs.python.org/library/xmlrpclib.html and http://phpxmlrpc.sourceforge.net/
• AMQP e.g. http://en.wikipedia.org/wiki/Advanced%5FMessage%5FQueuing%5FProtocol

Thrift is probably more what you're looking for. It's used by Facebook internally.