Creating multiple thread in JVM to fulfil high user load

Question

I have a C program that will be storing and retrieving alot of data in a Java store. I am putting alot of stress in my C program and multiple threads are adding and retrieving data from Java store. How will java handle such load? Because if there is only one main thread running JVM and handling all the requests from C, then it may become bottleneck for me. Will Java create multiple threads to handle the load, or is it programmers job to create and later on abort the threads?

My java store is just a Hashtable that stores the data from C, as is, against a key provided.

Answer 1

Plain Java requires that you write your own threading.

If you are communicating to java via web services it's likely that the web container will manage threads for you.

I guess you are using JNI, so then the situation is potentially more complex. Depending upon exactly how you are doing your JNI calls you can get at multiple threads in the JVM.

I've got to ask ... JNI is pretty gnarly and error prone, all too easy to bring down the whole process and get all manner of mysterious errors. Are there not C libraries containing a HashTable you could use? Or even write one, it's got to be less work than doing JNI.

Answer 2

I think this depends on the java code's implementation. If it proves not to thread, here's a potentially cleaner alternative to messy JNI:

Create a Java daemon process that communicates with your store, which INTERNALLY is threaded on requests, to guarantee efficient load handling. Use a single ExecutorService created by java.util.concurrent.Executors to service a work queue of store/retrieve operations. Each store/retrieve method call submits a Callable to the work queue and waits for it to be run. The ExecutorService will automagically queue and multithread the store/retrieval operations. This whole thing should be less than 100 lines of code, aside from communications with the C program.

You can communicate with this Java daemon from C using inter-process communication techniques (probably a socket), which would avoid JNI and let one Java daemon thread service numerous instances of the C program.

Alternately, you could use JNI to call the basic store/retrieve operations on your daemon. Same as currently, except the java daemon can decorate methods to provide caching, synchronization, and all sorts of fancy goodies associated with threading.

Answer 3

You definitely want to check the jni documentation about threading, which has information around attaching multiple native threads to the JVM. Also you should consider which Map implementation that you need to use. If accessing from multiple Hashtable will work, but may introduce a bottle neck as it is synchronized on every call, which will effectively mean a single thread reading or writing at a time. Consider the ConcurrentHashMap, which uses lock striping and providers better concurrent throughput.

A couple of things to consider if you are concerned about bottlenecks and latency.

• On a heavily loaded system, locking can introduce a high overhead. If the size of you map and the frequency of write allows, consider using an immutable map and perform a copy on write approach, where a single thread will handle writes by making updates to a copy of the map and replacing the original with the new version (make sure the reference is a volatile variable). This will allow reads to occur without blocking.
• Calling from C to Java via JNI will probably become a bottle neck too, its not as fast as calling in the other direction (Java to C). You can pass Direct ByteBuffers through to Java that contain references to the C data structures and allow Java to call back down to C via the Direct ByteBuffer.