Creating memory stream from ResponseStream Out of Memory Exception

Question

I am making a call to a httprequest which returns a pdf file in the responsestream. This works well for smaller pdf's, but not the file is up around 25-30MB, it is returning an out of memory exception.

        MemoryStream memStream = new MemoryStream();
        byte[] buffer = new byte[2048];

        int bytesRead = 0;
        do
        {
            bytesRead = _ResponseStream.Read(buffer, 0, buffer.Length);
            memStream.Write(buffer, 0, bytesRead);
        } while (bytesRead != 0);

        _ResponseStream.Close();
        _ResponseStream.Dispose();
        _ResponseStream = null;

        if (memStream.Length > 0)
            return memStream;
        else return null;

Machine specs are Quad Core 3.0GZ, 4GB RAM (Physical Box). It is possible that the PDF could reach sizes of 60MB. Multiple users can download at thte same time. How would this effect it? Would it be better to take the response stream straight to a FileStream? Is there a best practice way of doing this?

Answer 1

That should be fine - 60MB is reasonably small for a MemoryStream. The capacity doubles each time it needs to, so you may have about 120MB of backing array, but again that shouldn't break a decent machine.

Are you sure the input data is no bigger than that? How much memory do you have? Can you find out the size of the response stream first? If you can create the MemoryStream with the right capacity to start with, that would help.

Note that a using statement is better for the handling of the response stream - calling Close and Dispose and then setting a variable to null is overkill.

Answer 2

As Jon notes, 60MB is not particularly large for a MemoryStream.

Are you disposing of the returned MemoryStream after you are finished using it? Perhaps the 60MB is a red herring and it just happens to run out of memory on a particular file that happens to be 60MB?

Answer 3

It seems that the memory stream was indeed running out of memory at a certain point. As the file eventually gets stored on the file system, I am now taking the original stream from the httpwebrequest, and saving it straight to a file.

Answer 4

overlooked something, please delete.