Hi.
I'm currently using shared memory for IPC between Java and C++ apps, but looking for a more convenient alternative.
Can someone advice better method, but with same performance speed?
Thanks!
+3
A:
It depends how you plan to have your apps interract. In the posix environment you have pipes, shared memory, sockets, semaphores and message queues. See this question: Comparing unix lixux IPC for more information.
What is the interaction model for your processess (i.e. client/server, producer-consumer, etc)?
From personal experience, I would suggest your best bet would be pipes (since they are just files to read and write bytes) or sockets (since both languages support them).
mikelong
2009-05-24 19:54:30
Main consumer and several dumb (but highly efficient) producers.
SyRenity
2009-05-24 20:13:56
How fast the pipes or sockets, when compared to shared memory?
SyRenity
2009-05-24 20:25:25
@SyRenity , pretty fast. Depends on how much you lock and what locking you use really.Also +1, named pipes are cool.
Aiden Bell
2009-05-24 21:43:53
Final question - what is faster? :) Named pipes or sockets?
SyRenity
2009-05-25 09:14:04
I would do a benchmark to test this on your platform. The performance will vary with the kernel implementation.
mikelong
2009-05-25 11:48:51
+2
A:
As mikelong said, this depends a lot on what you are doing. AFAIK, none of the IPC methods have native Java bindings, so you're probably going to have to use JNI and make bindings yourself, so all the different methods are roughly equally as hard. If you are doing message passing though, I highly suggest using message queues. They are very easy to use (once you have the bindings), and have good performance. If you need to "share" some resource, then you probably want to stick with shared memory.
As it sounds like you are having some sort client/server thing, I would say use either message queues, unix domain sockets, or named pipes. They all involve copying data in the kernel, so they are not quite as fast as shared memory, but they are still very fast. If you have message-like data (individual small packets), go with message queues. That is probably the cleanest solution. If you have more of a stream of data, use pipes or sockets. Sockets have the advantage that you can easily make it network transparent (like X11) later on if you want, but they are slightly harder to work with than pipes. The performance is probably very similar.
Zifre
2009-05-24 19:59:57
I'm actually using JNA for IPC and it works fine - but again, the general inconvenience of shared memory (segments leftover, etc...) are a killer.
SyRenity
2009-05-24 20:12:05
Does Java support Unix domain sockets? You may need an add-on lib for that. If your data consists of a few fixed structures go with message queues. They are fast and easy to use. If the data is more variable and stream-like then go with named pipes.
Duck
2009-05-25 02:02:22
named pipes (fifos) are going to be slower than shared memory but that doesn't mean they aren't within tolerable limits for your app. Since you have multiple writers and one reader you are probably doing a lot of locking now. If your data is less than PIPE_BUF (usually around 4k) then your writes from various processes will be atomic and managed by the OS so that complexity is removed from your pgms. Alternatively you can have each writer use its own FIFO and multiplex your reader with something like select().
Duck
2009-05-25 14:53:37
Actually I use a circular buffer in shared memory, which allows me to not require any locking at all.AFAIK this possible only in shared memory - but perhaps mmap can do it it?Dunno about pipes or sockets at this department.
SyRenity
2009-05-25 17:37:52
A:
While likely not the most efficient, Java only supports sockets out of the box (the best I recall). They are very flexible, just perhaps not as fast as other options. As Zifre mentioned, it leaves you with an opportunity for network transparency, as well as language/binding transparency; as pretty much every language supports a socket library out of the box these days.
While I am throwing efficiency out of the window, if you wish to take it to the next level, you could probably wrap this in a Web Service of some kind. Use an embedded web server on the consumer for the producers to submit their data to.
monceaux
2009-05-25 08:25:48
A:
The easiest way to communicate between applications written in different languages is IMHO CORBA. There are very good open source CORBA ORBs out there. We use TAO for C++ and JacORB for Java. There are also companies like OCI and Remedy that provide technical support.
lothar
2009-05-26 20:35:49
CORBA would be overkill in this situation IMO. Its easy to forget how much work goes into setting up the ORBs, learning CORBA, IDL, etc. It's a daunting prospect for someone who just wants to pass some data on the same machine, much of it unstructured which is a bit of a PITA in CORBA.
Duck
2009-05-26 22:02:04
@Duck in my experience it's easier to learn how to use CORBA than to do cross language/platform communication manually correctly. Just to get all the marhalling/demarshalling right can be a nightmare. Not to mention all the nice services (naming, event, ...) that one can use out of the box.
lothar
2009-05-26 23:26:14