How do a send an HTTPS request through a proxy in Java?

Question

I am trying to send a request to a server using the HttpsUrlConnection class. The server has certificate issues, so I set up a TrustManager that trusts everything, as well as a hostname verifier that is equally lenient. This manager works just fine when I make my request directly, but it doesn't seem to be used at all when I send the request through a proxy.

I set my proxy settings like this:

Properties systemProperties = System.getProperties();
systemProperties.setProperty( "http.proxyHost", "proxyserver" );
systemProperties.setProperty( "http.proxyPort", "8080" );
systemProperties.setProperty( "https.proxyHost", "proxyserver" );
systemProperties.setProperty( "https.proxyPort", "8080" );

The TrustManager for the default SSLSocketFactory is set up like this:

SSLContext sslContext = SSLContext.getInstance( "SSL" );

// set up a TrustManager that trusts everything
sslContext.init( null, new TrustManager[]
    {
        new X509TrustManager()
        {
            public X509Certificate[] getAcceptedIssuers()
            {
                return null;
            }

            public void checkClientTrusted( X509Certificate[] certs, String authType )
            {
                // everything is trusted
            }

            public void checkServerTrusted( X509Certificate[] certs, String authType )
            {
                // everything is trusted
            }
        }
    }, new SecureRandom() );

// this doesn't seem to apply to connections through a proxy
HttpsURLConnection.setDefaultSSLSocketFactory( sslContext.getSocketFactory() );

// setup a hostname verifier that verifies everything
HttpsURLConnection.setDefaultHostnameVerifier( new HostnameVerifier()
{
    public boolean verify( String arg0, SSLSession arg1 )
    {
        return true;
    }
} );

If I run the following code, I end up with an SSLHandshakException ("Remote host closed connection during handshake"):

URL url = new URL( "https://someurl" );

HttpsURLConnection connection = (HttpsURLConnection)url.openConnection();
connection.setDoOutput( true );

connection.setRequestMethod( "POST" );
connection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded" );
connection.setRequestProperty( "Content-Length", "0" );

connection.connect();

I assume I am missing some kind of setting having to do with using a proxy when dealing with SSL. If I don't use a proxy, my checkServerTrusted method gets called; this is what I need to happen when I am going through the proxy as well.

I don't usually deal with Java and I don't have much experience with HTTP/web stuff. I believe I have provided all the detail necessary to understand what I am trying to do. If this isn't the case, let me know.

Update:

After reading the article suggested by ZZ Coder, I made the following changes to the connection code:

HttpsURLConnection connection = (HttpsURLConnection)url.openConnection();
connection.setSSLSocketFactory( new SSLTunnelSocketFactory( proxyHost, proxyPort ) );

connection.setDoOutput( true );
connection.setRequestMethod( "POST" );
connection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded" );
connection.setRequestProperty( "Content-Length", "0" );

connection.connect();

The result (SSLHandshakeException) is the same. When I set the SLLSocketFactory here to the SSLTunnelSocketFactory (the class explained in the article), the stuff I did with the TrustManager and the SSLContext is overridden. Don't I still need that?

Another Update:

I modified the SSLTunnelSocketFactory class to use the SSLSocketFactory that uses my TrustManager that trusts everything. It doesn't appear that this has made any difference. This is the createSocket method of SSLTunnelSocketFactory:

public Socket createSocket( Socket s, String host, int port, boolean autoClose )
    throws IOException, UnknownHostException
{
    Socket tunnel = new Socket( tunnelHost, tunnelPort );

    doTunnelHandshake( tunnel, host, port );

    SSLSocket result = (SSLSocket)dfactory.createSocket(
        tunnel, host, port, autoClose );

    result.addHandshakeCompletedListener(
        new HandshakeCompletedListener()
        {
            public void handshakeCompleted( HandshakeCompletedEvent event )
            {
                System.out.println( "Handshake finished!" );
                System.out.println(
                    "\t CipherSuite:" + event.getCipherSuite() );
                System.out.println(
                    "\t SessionId " + event.getSession() );
                System.out.println(
                    "\t PeerHost " + event.getSession().getPeerHost() );
            }
        } );

    result.startHandshake();

    return result;
}

When my code calls connection.connect, this method is called, and the call to doTunnelHandshake is successful. The next line of code uses my SSLSocketFactory to create an SSLSocket; the toString value of result after this call is:

"1d49247[SSL_NULL_WITH_NULL_NULL: Socket[addr=/proxyHost,port=proxyPort,localport=24372]]".

This is meaningless to me, but it might be the reason things break down after this.

When result.startHandshake() is called, the same createSocket method is called again from, according to the call stack, HttpsClient.afterConnect, with the same arguments, except Socket s is null, and when it comes around to result.startHandshake() again, the result is the same SSLHandshakeException.

Am I still missing an important piece to this increasingly complicated puzzle?

This is the stack trace:

javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:808)
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1112)
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1139)
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1123)
  at gsauthentication.SSLTunnelSocketFactory.createSocket(SSLTunnelSocketFactory.java:106)
  at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:391)
  at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:166)
  at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:133)
  at gsauthentication.GSAuthentication.main(GSAuthentication.java:52)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
  at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)
  ... 8 more

Answer 1

HTTPS proxy doesn't make sense because you can't terminate your HTTP connection at the proxy for security reasons. With your trust policy, it might work if the proxy server has a HTTPS port. Your error is caused by connecting to HTTP proxy port with HTTPS.

You can connect through a proxy using SSL tunneling (many people call that proxy) using proxy CONNECT command. However, Java doesn't support newer version of proxy tunneling. In that case, you need to handle the tunneling yourself. You can find sample code here,

http://www.javaworld.com/javaworld/javatips/jw-javatip111.html

EDIT: If you want defeat all the security measures in JSSE, you still need your own TrustManager. Something like this,

 public SSLTunnelSocketFactory(String proxyhost, String proxyport){
      tunnelHost = proxyhost;
      tunnelPort = Integer.parseInt(proxyport);
      dfactory = (SSLSocketFactory)sslContext.getSocketFactory();
 }

 ...

 connection.setSSLSocketFactory( new SSLTunnelSocketFactory( proxyHost, proxyPort ) );
 connection.setDefaultHostnameVerifier( new HostnameVerifier()
 {
    public boolean verify( String arg0, SSLSession arg1 )
    {
        return true;
    }
 }  );

EDIT 2: I just tried my program I wrote a few years ago using SSLTunnelSocketFactory and it doesn't work either. Apparently, Sun introduced a new bug sometime in Java 5. See this bug report,

http://bugs.sun.com/view%5Fbug.do?bug%5Fid=6614957

The good news is that the SSL tunneling bug is fixed so you can just use the default factory. I just tried with a proxy and everything works as expected. See my code,

public class SSLContextTest {

    public static void main(String[] args) {

     System.setProperty("https.proxyHost", "proxy.xxx.com");
     System.setProperty("https.proxyPort", "8888");

     try {

      SSLContext sslContext = SSLContext.getInstance("SSL");

      // set up a TrustManager that trusts everything
      sslContext.init(null, new TrustManager[] { new X509TrustManager() {
       public X509Certificate[] getAcceptedIssuers() {
        System.out.println("getAcceptedIssuers =============");
        return null;
       }

       public void checkClientTrusted(X509Certificate[] certs,
         String authType) {
        System.out.println("checkClientTrusted =============");
       }

       public void checkServerTrusted(X509Certificate[] certs,
         String authType) {
        System.out.println("checkServerTrusted =============");
       }
      } }, new SecureRandom());

      HttpsURLConnection.setDefaultSSLSocketFactory(
        sslContext.getSocketFactory());

      HttpsURLConnection
        .setDefaultHostnameVerifier(new HostnameVerifier() {
         public boolean verify(String arg0, SSLSession arg1) {
          System.out.println("hostnameVerifier =============");
          return true;
         }
        });

      URL url = new URL("https://www.verisign.net");
      URLConnection conn = url.openConnection();
            BufferedReader reader = 
                new BufferedReader(new InputStreamReader(conn.getInputStream()));
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
     } catch (Exception e) {
      e.printStackTrace();
     } 
    }
}

This is what I get when I run the program,

checkServerTrusted =============
hostnameVerifier =============
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"&gt;
......

As you can see, both SSLContext and hostnameVerifier are getting called. HostnameVerifier is only involved when the hostname doesn't match the cert. I used "www.verisign.net" to trigger this.

Answer 2

Try the Apache Commons HttpClient library instead of trying to roll your own: http://hc.apache.org/httpclient-3.x/index.html

From their sample code:

  HttpClient httpclient = new HttpClient();
  httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);

  /* Optional if authentication is required.
  httpclient.getState().setProxyCredentials("my-proxy-realm", " myproxyhost",
   new UsernamePasswordCredentials("my-proxy-username", "my-proxy-password"));
  */

  PostMethod post = new PostMethod("https://someurl");
  NameValuePair[] data = {
     new NameValuePair("user", "joe"),
     new NameValuePair("password", "bloggs")
  };
  post.setRequestBody(data);
  // execute method and handle any error responses.
  // ...
  InputStream in = post.getResponseBodyAsStream();
  // handle response.


  /* Example for a GET reqeust
  GetMethod httpget = new GetMethod("https://someurl");
  try { 
    httpclient.executeMethod(httpget);
    System.out.println(httpget.getStatusLine());
  } finally {
    httpget.releaseConnection();
  }
  */

Answer 3

See also this file upload tutorial using Apache HttpComponentsClient 4.0

http://radomirmladenovic.info/2009/02/13/file-upload-with-httpcomponents-successor-of-commons-httpclient