HttpURLConnection implementation question

Question

I have read that HttpURLConnection supports persistent connections, so that a connection can be reused for multiple requests. I tried it and the only way to send a second POST was by calling openConnection for a second time. Otherwise I got a IllegalStateException("Already connected"); I used the following:

try{
URL url = new URL("http://someconection.com");
}
catch(Exception e){}
HttpURLConnection con = (HttpURLConnection) url.openConnection();
//set output, input etc
//send POST
//Receive response
//Read whole response
//close input stream
con.disconnect();//have also tested commenting this out
con = (HttpURLConnection) url.openConnection();
//Send new POST

The second request is send over the same TCP connection (verified it with wireshark) but I can not understand why (although this is what I want) since I have called disconnect. I checked the source code for the HttpURLConnection and the implementation does keep a keepalive cache of connections to the same destinations. My problem is that I can not see how the connection is placed back in the cache after I have send the first request. The disconnect closes the connection and without the disconnect, still I can not see how the connection is placed back in the cache. I saw that the cache has a run method to go through over all idle connections (I am not sure how it is called), but I can not find how the connection is placed back in the cache. The only place that seems to happen is in the finished method of httpClient but this is not called for a POST with a response. Can anyone help me on this?

EDIT My interest is, what is the proper handling of an HttpUrlConnection object for tcp connection reuse. Should input/output stream be closed followed by a url.openConnection(); each time to send the new request (avoiding disconnect())? If yes, I can not see how the connection is being reused when I call url.openConnection() for the second time, since the connection has been removed from the cache for the first request and can not find how it is returned back. Is it possible that the connection is not returned back to the keepalive cache (bug?), but the OS has not released the tcp connection yet and on new connection, the OS returns the buffered connection (not yet released) or something similar? EDIT2 The only related i found was from [link text][1]

...when the application calls close() on the InputStream returned by URLConnection.getInputStream(), the JDK's HTTP protocol handler will try to clean up the connection and if successful, put the connection into a connection cache for reuse by future HTTP requests.

But I am not sure which handler is this. sun.net.www.protocol.http.Handler does not do any caching as I saw Thanks!

[1]: http://JDK Keep-Alive

[1]: http://download.oracle.com/javase/1.5.0/docs/guide/net/http-keepalive.html JDK_KeepAlive

Answer 1

From the javadoc for HttpURLConnection (my emphasis):

Each HttpURLConnection instance is used to make a single request but the underlying network connection to the HTTP server may be transparently shared by other instances. Calling the close() methods on the InputStream or OutputStream of an HttpURLConnection after a request may free network resources associated with this instance but has no effect on any shared persistent connection. Calling the disconnect() method may close the underlying socket if a persistent connection is otherwise idle at that time.

Answer 2

Should input/output stream be closed followed by a url.openConnection(); each time to send the new request (avoiding disconnect())?

Yes.

If yes, I can not see how the connection is being reused when I call url.openConnection() for the second time, since the connection has been removed from the cache for the first request and can not find how it is returned back.

You are confusing the HttpURLConnection with the underlying Socket and its underlying TCP connection. They aren't the same. The HttpURLConnection instances are GC'd, the underlying Socket is pooled, unless you call disconnect().

Answer 3

I found that the connection is indeed cached when the InputStream is closed. Once the inputStream has been closed the underlying connection is buffered. The HttpURLConnection object is unusable for further requests though, since the object is considered still "connected", i.e. its boolean connected is set to true and is not cleared once the connection is placed back in the buffer. So each time a new HttpUrlConnection should be instantiated for a new POST, but the underlying TCP connection will be reused, if it has not timed out. So EJP answer's was the correct description. May be the behavior I saw, (reuse of the TCP connection) despite explicitly calling disconnect() was due to caching done by the OS? I do not know. I hope someone who knows can explain. Thanks.

Answer 4

Hmmh. I may be missing something here (since this is an old question), but as far as I know, there are 2 well-known ways to force closing of the underlying TCP connection:

• Force use of HTTP 1.0 (1.1 introduced persistent connections) -- this as indicated by the http request line
• Send 'Connection' header with value 'close'; this will force closing as well.