I two programs running, one in Python and one in C++, and I need to share a two-dimensional array (just of decimal numbers) between them. I am currently looking into serialization, but pickle is python-specific, unfortunately. What is the best way to do this?
Thanks
Edit: It is likely that the array will only have 50 elements or so, but the transfer of data will need to occur very frequently: 60x per second or more.
You could try using boost::python to make your applications interoperable.
Some information about pickle support and plain boost::python documentation.
You could try hosting the array in a Memory-mapped file, although you will need to synchronize access to the file to avoid race conditions.
Alternatively you could establish a socket (or pipe) connection between both processes and pass values by exchanging messages.
How large is this array? If it isn't very large, then JSON serialization is a good fit. There are libraries readily available for C++, and Python has JSON serialization in its standard library as of version 2.6. See http://www.json.org/ for more info.
I would propose simply to use c arrays(via ctypes on the python side) and simply pull/push the raw data through an socket
Serialization is one problem while IPC is another. Do you have the IPC portion figured out? (pipes, sockets, mmap, etc?)
On to serialization - if you're concerned about performance more than robustness (being able to plug more modules into this architecture) and security, then you should take a look at the struct module. This will let you pack data into C structures using format strings to define the structure (takes care of padding, alignment, and byte ordering for you!) In the C++ program, cast a pointer to the buffer to the corresponding structure type.
This works well with a tightly-coupled Python script and C++ program that is only run internally.