views:

282

answers:

4

Example:

  1. If I read a file and copy it to another file through .NET Streams will the total size of the file occupy the memory at any moment? Or will the bytes be discarded as soon as they are used?
  2. If the naive approach doesn't saves memory would buffered Streams do it?
+12  A: 

If you stream your copying, i.e. read a buffer, write a buffer, read a buffer, write a buffer etc until you've run out of data, it will only take as much memory as the buffer size. I expect File.Copy to do this (in the native Windows code, admittedly).

If you want to do it yourself, use something like this:

public void CopyData(Stream input, Stream output)
{
    byte[] buffer = new byte[32 * 1024];
    int read;
    while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
    {
        output.Write(buffer, 0, read);
    }
}

This will only take 32K however big the stream is.

EDIT: As noted in the comments, the streams may well have their own buffers as well, but the point is that you could still transfer a very large file without running out of memory.

Jon Skeet
... + the size of any internal buffers used by the Stream (FileStream uses an internal buffer whose size defaults to 4K)
Joe
.NET FileStreams have their own internal buffers. So using the above code with FileStreams probably takes slightly more than 32K of memory (32K + whatever the size of FileStream's internal buffer is). But in any case, a constant amount of memory is used with this technique.
C. Dragon 76
Good points, both of you. Updating...
Jon Skeet
+3  A: 

If you call something like ReadToEnd() then yes, the contents of the file will be loaded into memory. You're correct in guessing that using a buffer is the appropriate approach to ensure that only a subset of the file's data is loaded into memory at any given time.

Ken Browning
+2  A: 

No, the whole file won't be loaded into memory.

The memory footprint will depend on the size of buffers you use for reading and writing, and any internal buffers maintained by the stream.

The FileStream class uses an internal buffer whose size can be specified in a constructor overload and which defaults to 0x1000 bytes (probably implementation dependent - this value was obtained by examining the FileStream class using Lutz Reflector).

Joe
+1  A: 

It really depends on your methodology. If you are using the stream a temporary endpoint, ala ReadToEnd(), you can load the entire file in memory. If, instead, you are buffering, you will not use more than a bit of overhead over the buffer size.

Gregory A Beamer
Brief note: ReadToEnd() exists on TextReader, not Stream.
Jon Skeet