Java UrlConnection HTTP 1.0

Question

I am trying to download a file from my Java application. But because UrlConnection uses HTTP 1.1 protocol i get a Tranfer Encoding: chunked response in which case i can not find out file size(content-length is not set). From what i could find HTTP version is hard coded in the class and there is no way to change it. Is it somehow possible to change the version back to one or tell the server not to use chunked encoding when sending a file?

Edit: I am not trying to retrive dynamic content my application is a download manager. files i am downloading are static. Other downloaders i checked wget,igetter,curl use Http 1.0 and they get the size info from most servers. But my application and firefox issuing Http 1.1 always gets chunked encoding. I understand that content-length is not always present but i would like to get it most of time.

Answer 1

Two ways I can think of:

• Use HTTPClient instead of UrlConnection, but you need to set the parameter HTTPClient.forceHTTP_1.0=true in order to enforce HTTP1.0
• Use TCP socket and implement http over it (not that difficult). I would personally go with this solution :P

Answer 2

The Jakarta Commons HTTP Client contains a "preference architecture" that allows some fine grained control over the particulars of the HTTP connection. See http://hc.apache.org/httpclient-3.x/preference-api.html

Answer 3

It's very likely that the server can't specify a valid content-length, even if you specify HTTP/1.0. When content is dynamically produced, the server has to buffer it all to measure its total length. Not all servers are going to be able to fallback to this less efficient behavior.

If buffering the response is reasonable, why not do it in your client, where you have full control? This is safer than relying on the server.

Read the response without processing, just stuffing the data into a ByteArrayOutputStream. When you are done, measure the length of the resulting byte array. Then create a ByteArrayInputStream with it and process that stream in place of the stream you got from the URLConnection.