Why I need to read file piece by piece to buffer?

Question

Hi, I have seen following code for getting the file into array, which is in turn used as a parameter for SQL command inserting it into a blob column:

using (FileStream fs = new FileStream(soubor,FileMode.Open,FileAccess.Read))

int length = (int)fs.Length;
buffer = new byte[length];
int count;
int sum = 0;
while ((count = fs.Read(buffer, sum, length - sum)) > 0)
    sum += count;

Why I cannot simply do that:

fs.Read(buffer, 0, length) in order to just copy content of file to the buffer?

Thanks

Answer 1

Because your file could be very large and the buffer has usually a fixed size of 4-32 KB. This way you know you're not filling your memory unnessecarily.

Of course, if you KNOW the size of your file is not too large or if you store the contents in memory anyways, there is no reason not to read it all in one shot.

Although, if you want to read the contents of your file directly into a variable, you don't need the Stream API. Rather use

File.ReadAllText(...)

or

File.ReadAllBytes(...)

Answer 2

There's more to it than just "the file may not fit in memory". The contract for Stream.Read explicitly says:

Implementations of this method read a maximum of count bytes from the current stream and store them in buffer beginning at offset. The current position within the stream is advanced by the number of bytes read; however, if an exception occurs, the current position within the stream remains unchanged. Implementations return the number of bytes read. The return value is zero only if the position is currently at the end of the stream. The implementation will block until at least one byte of data can be read, in the event that no data is available. Read returns 0 only when there is no more data in the stream and no more is expected (such as a closed socket or end of file). An implementation is free to return fewer bytes than requested even if the end of the stream has not been reached.

Note the last sentence - you can't rely on a single call to Stream.Read to read everything.

The docs for FileStream.Read have a similar warning:

The total number of bytes read into the buffer. This might be less than the number of bytes requested if that number of bytes are not currently available, or zero if the end of the stream is reached.

For a local file system I don't know for sure whether this will ever actually happen - but it could do for a network mounted file. Do you want your app to brittle in that way?

Reading in a loop is the robust way to do things. Personally I prefer not to require the stream to support the Length property, either:

public static byte[] ReadFully(Stream stream)
{
    byte[] buffer = new byte[8192];
    using (MemoryStream tmpStream = new MemoryStream())
    {
        int bytesRead;
        while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
        {
            tmpStream.Write(buffer, 0, bytesRead);
        }
        return tmpStream.ToArray();
    }
}

That is slightly less efficient when the length is known beforehand, but it's nice and simple. You only need to implement it once, put it in a utility library, and call it whenever you need to. If you really mind the efficiency loss, you could use CanSeek to test whether the Length property is supported, and repeatedly read into a single buffer in that case. Be aware of the possibility that the length of the stream could change while you're reading though...

Of course, File.ReadAllBytes will do the trick even more simply when you only need to deal with a file rather than a general stream.