I would like to chain multiple stream operations (like downloading a file, uncompressing it on the fly, and processing the data without any temp files). The files are in 7z format. There is a LZMA SDK available, but forces me to create an outside output stream instead of being a stream itself - in other words the output stream will have to be fully written before I can work with it. SevenZipSharp also seems to be missing this functionality.
Has anyone done something like that?
// in pseudo-code - CompressedFileStream derives from Stream
foreach (CompressedFileStream f in SevenZip.UncompressFiles(Web.GetStreamFromWeb(url))
{
Console.WriteLine("Processing file {0}", f.Name);
ProcessStream( f ); // further streaming, like decoding, processing, etc
}
Each file stream would behave like a read-once stream representing one file, and calling MoveNext() on the main compressed stream would automatically invalidate & skip that file.
Similar constructs can be done for compression. Example usage - do some aggregation on a very large quantity of data - for each 7z file in a dir, for each file inside, for each data line in each file, sum up some value.