SSLSockets and problem with Cipher Specs

Question

Hi

I'm trying to get my java client to communicate with a C server using SSL.

The problem is - I don't have any server sources and I'm getting a handshake failure error:

Exception in thread "main" javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:136)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1657)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:932)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1096)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1123)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1107)
        at posslu.Main.main(Main.java:36)
Java Result: 1

After some heavy duty wiresharking I now know, that working C clients ( I don't have access to their source code neither ) have these Cipher specs in client-server hello packets:

Cipher Spec: SSL2_DES_192_EDE3_CBC_WITH_MD5 (0x0700c0)
Cipher Spec: SSL2_IDEA_128_CBC_WITH_MD5 (0x050080)
Cipher Spec: SSL2_RC2_CBC_128_CBC_WITH_MD5 (0x030080)
Cipher Spec: SSL2_RC4_128_WITH_MD5 (0x010080)
Cipher Spec: SSL2_RC4_64_WITH_MD5 (0x080080)
Cipher Spec: SSL2_DES_64_CBC_WITH_MD5 (0x060040)
Cipher Spec: SSL2_RC2_CBC_128_CBC_WITH_MD5 (0x040080)
Cipher Spec: SSL2_RC4_128_EXPORT40_WITH_MD5 (0x020080)

and all packets are sent in SSLv2 protocol.

And these are the specs from SSLSocket.getSupportedCipherSuites() method:

SSL_RSA_WITH_RC4_128_MD5   
SSL_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_DSS_WITH_AES_128_CBC_SHA
SSL_RSA_WITH_3DES_EDE_CBC_SHA
SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA
SSL_RSA_WITH_DES_CBC_SHA
SSL_DHE_RSA_WITH_DES_CBC_SHA
SSL_DHE_DSS_WITH_DES_CBC_SHA
SSL_RSA_EXPORT_WITH_RC4_40_MD5
SSL_RSA_EXPORT_WITH_DES40_CBC_SHA
SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
SSL_RSA_WITH_NULL_MD5
SSL_RSA_WITH_NULL_SHA
SSL_DH_anon_WITH_RC4_128_MD5
TLS_DH_anon_WITH_AES_128_CBC_SHA
SSL_DH_anon_WITH_3DES_EDE_CBC_SHA
SSL_DH_anon_WITH_DES_CBC_SHA
SSL_DH_anon_EXPORT_WITH_RC4_40_MD5
SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA
TLS_KRB5_WITH_RC4_128_SHA
TLS_KRB5_WITH_RC4_128_MD5
TLS_KRB5_WITH_3DES_EDE_CBC_SHA
TLS_KRB5_WITH_3DES_EDE_CBC_MD5
TLS_KRB5_WITH_DES_CBC_SHA
TLS_KRB5_WITH_DES_CBC_MD5
TLS_KRB5_EXPORT_WITH_RC4_40_SHA
TLS_KRB5_EXPORT_WITH_RC4_40_MD5
TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA
TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5

Now I want to use

Cipher Spec: SSL2_RC4_128_WITH_MD5 (0x010080)

as it seems it is supported both by C server and my java client. So I came up with this code:

SSLSocketFactory sslsockfact = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket sslsocket = (SSLSocket) sslsockfact.createSocket( args[0], args[1] );

sslsocket.setEnabledCipherSuites( new String[] { "SSL_RSA_WITH_RC4_128_MD5" } );
sslsocket.startHandshake();

But for some reason these cipher specs are sent in client hello packet:

Cipher Spec: TLS_RSA_WITH_RC4_128_MD5 (0x000004)
Cipher Spec: SSL2_RC4_128_WITH_MD5 (0x010080)

Shouldn't there be only the second one included? This causes the server to send the server-hello packet in TLSv1 protocol and gets me a handshake failure.

I went on trying to figure out what's happening:

These are supported protocols I got from getSupportedProtocols():

SSLv2Hello
SSLv3
TLSv1

If I put something like this in my code:

sslsocket.setEnabledProtocols( new String[] { "SSLv2Hello" } );

It says:

Exception in thread "main" java.lang.IllegalArgumentException: SSLv2Hellocannot be enabled unless TLSv1 or SSLv3 is also enabled

If I switch to:

sslsocket.setEnabledProtocols( new String[] { "SSLv2Hello", "SSLv3" } );

server answers in SSLv3 and I get handshake failure...

So that's it I have no idea how to get this to work, any help?

Is it possible, that SSLv2 is no longer supported and I simply can not use it with java?

Answer 1

From this question I get:

"After you have installed the JSSE, you must set a system property and add a new security provider to the Security class object. There are a variety of ways to do both of these things, but for the purposes of this article, the programmatic method is shown:

System.setProperty( "java.protocol.handl er.pkgs", "com.sun.net.ssl.in ternal.www.protocol" ); Security.addProvide r(new com.sun.net.ssl.int ernal.ssl.Provider() );"

Hope that helps.