LDAP requests hanging for 15 seconds

Question

I have JBoss application server that is using LDAP for authentication. Lately we have noticed that there are a lot of slow requests (> 15 seconds).

I did some threaddumps of the server and noticed that many threads where waiting on a lock: com.sun.jndi.ldap.LdapRequest@54ceac

java.lang.Object.wait(Native Method)
com.sun.jndi.ldap.Connection.readReply(Connection.java:418)
com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:340)
com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:192)
com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2637)
com.sun.jndi.ldap.LdapCtx.(LdapCtx.java:283)
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:134)
com.sun.jndi.url.ldap.ldapURLContextFactory.getObjectInstance(ldapURLContextFactory.java:35)
javax.naming.spi.NamingManager.getURLObject(NamingManager.java:584)

All of the requests I have seen that have been waiting in this state have used more than 15 seconds to complete. We are monitoring the LDAP-server and all requests from the monitoring tool finish in less than 200 ms. This makes me think this is a problem with the com.sun.jndi.ldap code. Decompiling the com.sun.jndi.ldap.Connection class (jdk1.5.0_12) I see this:

BerDecoder readReply(LdapRequest ldaprequest) throws IOException, NamingException
{
_L2:
    BerDecoder berdecoder;
    if((berdecoder = ldaprequest.getReplyBer()) != null)
    break; /* Loop/switch isn't completed */
    try
    {
label0:
    {
        synchronized(this)
        {
     if(sock == null)
         throw new ServiceUnavailableException((new StringBuilder()).append(host).append(":").append(port).append("; socket closed").toString());
        }
        synchronized(ldaprequest)
        {
     berdecoder = ldaprequest.getReplyBer();
     if(berdecoder == null)
     {
         ldaprequest.wait(15000L);
         break label0;
     }
        }
        break; /* Loop/switch isn't completed */
    }
    }
    ...

There is apparently a hardcoded timeout of 15000 milliseconds.

Do anyone have any ideas for a fix/workaround?

Answer 1

looks to me as though it only waits if the reply is null - wondering if there is some kind of version mismatch that is causing your app not be able to parse the reply from your server.

Have you tried attaching source, and seeing you can set a breakpoint in eclipse.

-ace

Answer 2

I've seen something like this before when using LDAP to connect to an ActiveDirectory box (in a network where there's more than one server). It ended up being a DNS issue and we just needed to flush our DNS cache ("ipconfig /flushdns" on a windows box). This may or may not be your issue, just thought it'd be worth a try.

Answer 3

Sounds similar to this bug, have you tried inspecting the network traffic with packet sniffer to check for this condition?