views:

851

answers:

7

From msdn:

http://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream.aspx

This class cannot be used to compress files larger than 4 GB.

Do you know any other implementations for .net without the 4 gb limit?

Thanks

NOTE: I really need to decompress a file in GZ format with content larger than 4gb. Do you know any code that can do that? Thanks

+1  A: 

Having a look around seems a lot of people have encountered this problem. This thread seems the most comprehensive

System.IO.Compressio.DeflateStream clarifications please

the only implementation I was able to find that seems to overcome this problem by using Zip64 is

Xceed Zip for .NET

however it is very expensive and not sure if it would suite your needs.

Edit:

There does seem to be quite a number of implementations of Zip64 for .NET but can't find any that are free.

Mark Davidson
That external thread is not helpful. It is filled with supposition, and wrong ones. The DeflateStream does not keep all the stream data in memory - that is not the reason for the 4gb limitation.
Cheeso
DotNetZip is a managed .NET library for ZIP files, that does ZIP64. It's free (gratis + libre), though donations are encouraged. ZIP64 is a solution to a different problem than the original poster asked about - the limitation of the BCL DeflateStream.
Cheeso
+6  A: 

There is sample code at CodeProject using the 7zip library.

The license is open so you should be able to use this in your project.

7-Zip also supports GZ files.

benPearce
Would up vote this if I had some votes left, this seems like a good one and looking at the change log does have Zip64 support.
Mark Davidson
Way better than SharpZipLib (which is a facade for zlib). 7zip is amazing.
RobS
+2  A: 

Take a look at SharpZipLib. Not sure if it's subject to the same limitation, but worth a look.

Dave Markle
+1  A: 

Look for libraries that support DEFLATE64 (not Zip64, that's an extension to the ZIP file format). Xceed Zip for .NET does support Deflate64, and I'm sure others do too.

Martin Plante
+5  A: 

FYI, we have removed the 4 GB limit from DeflateStream in .NET 4.

Justin Van Patten
cool! I'm waiting for that ;)
Luca Martinetti
But someone like me who is deflating gzipped files in SSIS is out of luck - you guys skipped releasing a new version with vs2010 / sql2008 r2 :(
Burg
A: 

DotNetZip does ZIP64 for .NET, and it is free. But Zip64 is not the same as Deflate64.

Cheeso
A: 

Although that documentation says the 4GB limitation is for both the DeflateStream and GZipStream, only GZipStream is limited because of the CRC32 checksum. If you do not need CRC32 then use DeflateStream.

Interesting. Couple questions. #1: Are you sure about this? #2. Why does the CRC32 limit the size of the data that can be compressed? #3. What is the source of your information? (are you familiar with the DeflateStream implementation, or what?)
Cheeso
Hi,1. I am absolutely sure - I have tested it.2. See Wikipedia: http://en.wikipedia.org/wiki/Cyclic_redundancy_check3. I used .NET Reflector (http://www.red-gate.com/products/reflector/) to peek at the code. The 4GB limitation is completely ignored if CRC32 is not used.