Java HTTP Requests Buffer size

Question

Hello,

I have an HTTP Request Dispatcher class that works most of the time, but I had noticed that it "stalls" when receiving larger requests. After looking into the problem, I thought that perhaps I wasn't allocating enough bytes to the buffer. Before, I was doing:

byte[] buffer = new byte[10000];

After changing it to 20000, it seems to have stopped stalling:

String contentType = connection.getHeaderField("Content-type");
            ByteArrayOutputStream baos = new ByteArrayOutputStream();

            InputStream responseData = connection.openInputStream();
            byte[] buffer = new byte[20000];
            int bytesRead = responseData.read(buffer);
            while (bytesRead > 0) {
                baos.write(buffer, 0, bytesRead);
                bytesRead = responseData.read(buffer);
            }
            baos.close();
            connection.close();

Am I doing this right? Is there anyway that I can dynamically set the number of bytes for the buffer based on the size of the request?

Thanks...

Answer 1

If you're open to using external libraries, the Apache IOUtils library has a toByteArray that will convert an input stream to a byte array without any work on your part.

It's as simple as:

byte[] buffer = IOUtils.toByteArray(connection.openInputStream());

Answer 2

This is the wrong way of streaming from input to output. The right way is:

byte[] buffer = new byte[10000];
int bytesRead = 0;
while ((bytesRead = responseData.read(buffer)) > 0) {
    baos.write(buffer, 0, bytesRead);
}

or

byte[] buffer = new byte[10000];
for (int bytesRead = 0; (bytesRead = responseData.read(buffer)) > 0;) {
    baos.write(buffer, 0, bytesRead);
}

Also see the Sun tutorial on the subject.

A buffer of 1~2K (1024~2048) is by the way often more than enough.