Java RMI timeout in callback

Question

We are using Java RMI for communication. An RMI client passes a processing request and an object with a callback method to an RMI server. The server invokes the callback when it is done with processing. The setup is similar to the one described in RMI Callbacks.

Occasionally we are getting a "read time out" exception in the server upon invoking the callback method. The callback thread stalls for about a minute before the exception is raised.

java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
    java.net.SocketTimeoutException: Read timed out
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
    at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:178)
    at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:132)
    at $Proxy2.finished(Unknown Source)
    at com.unrisk.db.grid.GridTask.invokeCallback(com.unrisk.db.grid.GridTask:1292)
    at com.unrisk.db.grid.GridTask.invokeCallbacks(com.unrisk.db.grid.GridTask:1304)
    at com.unrisk.db.service.tasks.EquityMDTask.afterRun(com.unrisk.db.service.tasks.EquityMDTask:276)
    at com.unrisk.db.grid.GridTask.run(com.unrisk.db.grid.GridTask:720)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
    at java.io.DataInputStream.readByte(DataInputStream.java:248)
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:228)
    ... 12 more

We are using Sun Java JDK 1.6.0_18 under Windows Server 2003 32-bit.

Is it possible to work around the connection problems by tuning RMI related system properties?

Answer 1

A lot of the timeouts are described in this bookmark-worthy post: http://archives.java.sun.com/cgi-bin/wa?A2=ind0208&L=jini-users&P=R27617

In my (limited) experience with JRMP, I found I had to use JVM-global properties to tweak the read timeouts...

digression: My team now uses Jeri (Java extensible remote invocation) exclusively, which is part of the Apache River (formerly: Sun Jini) project. Jeri is a lot more configurable than JRMP, allowing you to define your own underlying protocols.