Web service SSL handshake fails in production environment unless SSL debugging enabled

Question

Scenario: calling a client web service over SSL (https) with mutual SSL authentication. Different service endpoint URLs and certs (both keystore and truststore) for test vs. production environments. Both test and production environments run tomcat / JBoss clustered. Production environment has load balancing / BigIP, runs Blade and non-Blade machines.

Truststore is set (using -Djavax.net.ssl.trustStore=value) at startup. Keystore is set using System.setProperty("javax.net.ssl.keyStore", "value") in Java code. Web service call made using Axis2. All works fine in test environment, but when we moved to production environment (6 servers), it appears certs are not being forwarded for the handshake. Here's what we've done:

• in test environment, handshake using test versions of certs has been working all along, with no ssl debugging enabled
• confirmed in test environment that handshake with client production endpoint succeeds (production certs, both ours and theirs, are fine) -- this was done using -Djavax.net.debug=handshake,ssl
• confirmed that the error condition occurs on all 6 production servers
• took one server out of the cluster, turned on ssl debugging for just that one (with a restart), hit it directly, handshake works!
• switched to a different server without the debugging turned on, handshake error condition occurs
• turned debugging on on that second server (with a restart), hit it directly, handshake works!

From the evidence, it seems like somehow the debugging being enabled causes the certificates to be properly retrieved/conveyed, although that makes no sense! I wonder whether somehow the enabled debugging makes the system pay attention to the System.setProperty call, and ignore it otherwise. However, in local and test environments, handshake worked without debugging enabled.

Do I maybe need to be setting keystore on server startup like I'm setting truststore? Have been avoiding that because the keystore will differ for each of our test environments (16 of them).

Answer 1

Turns out that the debug setting was a red herring. What actually bit us was that there was an existing client with an SSL/basic authentication web service we call when one of their users logs in. Since in that context the keystore wasn't relevant, the javax.net.ssl.keyStore property doesn't get set -- but the SSL exchange still tries to load a keyStore (which ends up not loading any certs). Since, unfortunately, even if the javax.net.ssl.keyStore value is changed, it does not get reloaded, calls to the other client's web service sent along no keystore certs.

The solution was to set the keyStore property at server startup rather than at the point of the web service call. If at some point in the future we need to be able to use different keyStores in different contexts, it looks like we'd need to implement a custom SocketFactory.