how can I avoid performance bottleneck when using jni in a java web application/service

Question

Greetings, I could not provide all the details in the question, so here are the key details.

I have a native dll (and a corresponding .so) wrapping a static library, which is created by Eiffel programming language. I've written a C++ wrapper around the static lib, and I've successfully exposed this to Java. However, if I use this dll in a web application, things will become kinda complicated. The problem is multiple java threads will access the same C++ code, where native context(?) is kept between calls by different classes & instances. In order to use functionality from the Eiffel code, one has to initialize Eiffel runtime from C++ and then use Eiffel libraries to make use of Eiffel classes from C++. Unfortunately, this means that all incoming requests to Java server side end up in a single location in C++ (within native dll), where there is only one Eiffel runtime. This situation forces me to make the whole Eiffel runtime thread safe, and only one operation by one Java thread can use Eiffel code, by passing through JNI. I have a feeling that this may quickly become a scalability issue.

I feel that I may need a pool of processes, each loading a copy of the same dll(or .so under *nix) which will be served to incoming threads from Java. So once the shared library is loaded, C++ code will create, say 10 processes, and incoming threads from Java side will be allocated to these processes via C++ code. The flow of events would be like this:

Java thread accesses native code (c++) in shared library. Native code checks which processes are available from the process pool makes use of one of the processes via ipc, marking it busy (probably using a thread?)

This is the only cross platform way I could think of to safely load the same piece of code (Eiffel runtime and eiffel classes) without any thread safety issues.

This is all due to Eiffel runtime being an expensive, and global component, which I must expose via JNI.

Or should I simply go with thread safe operations, where only one thread is served from JNI at any time? Is there any trick I can do with Java to create lightweight isolated jvm like containers each using only a single Eiffel runtime?

Your feedback would be much appreciated.

Best Regards Seref

Answer 1

First question: if you had multiple copies of the DLL loaded in separate processes would they operate correctly? Assuming "yes" then your idea of having multiple copies running sounds to have potential. (Actually, there is another question to ask first: I'm assuming that re-implementing the Eiffel in Java has already been examined and proven to be too difificult?)

My inclination would first be to check whether your simple, single, thread-safe approach might give good enough performance. If it doesn't and you need scalability then another possibility is to consider using several cheap meachine (or virtual machines) - this has the great merit of simplicity. It doesn't take much development effort to cost more than a few machines.

Otherwise your idea of a pool of "service" processes sounds like a reasonable idea. There's plenty of different Inter-process communication (IPC) possibilities. Almost by definition you're spending a long time in the service in each call (otherwise would you have a problem?) so in which case the actual IPC mechanism need not be highly optimised - favour simplicity and ease of administration. I would look first at a queue-based approach using JMS - yes there are plenty of other options including RMI or low-level sockets - but JMS has two advantages: it's very easy and scalability kind of falls out, the requester just pops messages on a queue, never needs to know how many service processes there may be. Also interestingly on some vendors platforms there are C++ implementations of JMS (XMS is the one I use) and hence you don't even meed Java in the service process.

To elaborate: the starting position is

  WS-Client ---WS Call ---> WS in Web App ---JNI--->C++/Eiffel

I suggest using

  WS-Client ---WS Call ---> WS in Web App --JMS enq--> Q --JMS deq--> Java---JNI--->C++/Eiffel

or

  WS-Client ---WS Call ---> WS in Web App --JMS enq--> Q --XMS deq--->C++/Eiffel

Responses can be returned on temparary response queues with the request message contains a Reply To Q name and correlation information.

As it happens this latter pattern is pretty much what my current app is doing, with nice performance. I think having no Java in the C++/Eiffel engine may be a win. If that pseudo synchronous use of JMS seems unappealing then my alternative would be to use EJBs with remote interfaces. The idea again is to push all the scalability work into the infrastructure.