Technique or utility to minimize Java "warm-up" time?

Question

I am supporting a Java messaging application that requires low latency (< 300 microseconds processing each message). However, our profiling shows that the Sun Java Virtual Machine runs slowly at first, and speeds up after the first 5,000 messages or so. The first 5,000 messages have latency of 1-4 milliseconds. After about the first 5,000, subsequent messages have ~250 microseconds latency, with occasional outliers.

It's generally understood that this is typical behavior for a Java application. However, from a business standpoint it's not acceptable to tell the customer that they have to wait for the JVM to "warm-up" before they see the performance they demand. The application needs to be "warmed-up" before the first customer message is processed

The JVM is Sun 1.6.0 update 4.

Ideas for overcoming this issue:

1. JVM settings, such as -XX:CompileThreshold=
2. Add a component to "warm-up" the application on startup, for example by sending "fake messages" through the application.
3. Statically load application and JDK classes upon application startup, so that classes aren't loaded from JARs while processing customer messages.
4. Some utility or Java agent that accomplishes either or both of the above two ideas, so that I don't have to re-invent the wheel.

NOTE: Obviously for this solution I'm looking at all factors, including chip arch, disk type and configuration and OS settings. However, for this question I want to focus on what can be done to optimize the Java application and minimize "warm up" time.

Answer 1

Your problem is not class loading but "just in time" compilation.

Try -XX:CompileThreshold=1

That will force Java to compile everything the first time it runs it. It will slow down the startup of your code somewhat but not VM code (since that gets compiled when Java is installed). There is a bug open to allow Java to compile custom JARs in a simular way and save the result for later executions which would greatly reduce this overhead but there is no pressure to fix this bug any time soon.

A second option would be to send 5'000 fake messages to the app to "warm it up". Sell this as "making sure everything is set up correctly".

[EDIT] Some background info in precompiling classes: Class Data Sharing

You may want to try IBM's version of Java since here, you can add more classes to the shared pool: Overview of class data sharing

Answer 2

Are you using the client or the server JVM? Try starting your program with:

java -server com.mycompany.MyProgram

When running Sun's JVM in this mode, the JIT will compile the bytecode to native code earlier; because of this, the program will take longer to start, but it will run faster after that.

Reference: Frequently Asked Questions About the Java HotSpot VM

Quote:

What's the difference between the -client and -server systems?

These two systems are different binaries. They are essentially two different compilers (JITs)interfacing to the same runtime system. The client system is optimal for applications which need fast startup times or small footprints, the server system is optimal for applications where the overall performance is most important. In general the client system is better suited for interactive applications such as GUIs. Some of the other differences include the compilation policy,heap defaults, and inlining policy.

Answer 3

"Warm-up" in Java is generally about two things:

(1): Lazy class loading: This can be work around by force it to load.

The easy way to do that is to send a fake message. You should be sure that the fake message will trigger all access to classes. For exmaple, if you send an empty message but your progrom will check if the message is empty and avoid doing certain things, then this will not work.

Another way to do it is to force class initialization by accessing that class when you program starts.

(2): The realtime optimization: At run time, Java VM will optimize some part of the code. This is the major reason why there is a warm-up time at all.

To ease this, you can sent bunch of fake (but look real) messages so that the optimization can finish before your user use it.

Another that you can help to ease this is to support inline such as using private and final as much as you can. the reason is that, the VM does not need to look up the inheritance table to see what method to actually be called.

Hope this helps.

Answer 4

Seems like your project would benefit from real time guarantees:

See: Real Time Java