OutOfMemoryException using BZip2 (SharpZipLib)

Question

I use Asp.net , .net 3.5, win2003, iis 6.0.

I use Oracle for gathering files, saving file in SharpZipLib.BZip2 compressed format in field RAW in table Oracle.

My application is Web, and I use WCF Service for get data (array of bytes) of a file. The aspx page send file to user (download file).

My issue-problem:

I read DATA from Oracle, (I call to WCF Service). I get array of bytes (byte[]),

I try Uncompress file using SharpZipLib.BZip2

using (MemoryStream inData = new MemoryStream(data))
{ 
using (MemoryStream outData = new MemoryStream())
{
          BZip2.Decompress(inData, outData); //<==================== Fails here OutOfMemoryException
          return outData.ToArray();
}

}

the error is because the file "uncompressed" is big, very big (> 500 MB) !!!

compressed file: 4MB

uncompressed file: > 500 MB

I do test like this:

BufferedStream bufin = new BufferedStream(instream);

            using (MemoryStream outData = new MemoryStream())
            {
                BZip2.Decompress(bufin, outData);

                return outData.ToArray();
            }

But I get the same OutOfMemoryException

Trace Stack of Exception

   en System.IO.MemoryStream.set_Capacity(Int32 value)
   en System.IO.MemoryStream.EnsureCapacity(Int32 value)
   en System.IO.MemoryStream.WriteByte(Byte value)
   en Reale.Frk.Compression.BZip2.BZip2.Decompress(Stream inStream, Stream outStream)

Code of SharpZipLib.BZip2.Decompress

public static void Decompress(Stream inStream, Stream outStream) 

            {

                  if ( inStream == null ) {

                        throw new ArgumentNullException("inStream");

                  }

                  if ( outStream == null ) {

                        throw new ArgumentNullException("outStream");

                  }


                  using ( outStream ) {

                        using ( BZip2InputStream bzis = new BZip2InputStream(inStream) ) {

                             int ch = bzis.ReadByte();

                             while (ch != -1) {

                                   outStream.WriteByte((byte)ch);

                                   ch = bzis.ReadByte();

                             }

                        }

                  }

            }

any suggestions, comments, sample source code ?

Answer 1

Skip the MemoryStream and write directly to a file.

Else add more memory to the server.

Another option to specify the initial capacity for the MemoryStream.