C# FileStream : Optimal buffer size for writing large files ?

Question

Suppose I'm writing a couple of files to disk, between 2MB and 5GB. What are sensible buffer values for the FileStream ?

Is it sensible to work with buffersizes of several megabytes, or should I stick to kilobyte-buffers ?

Answer 1

Default buffer size is 8 kbytes.

Also, take a look here: Sequential File Programming Patterns and Performance with .NET

Answer 2

My guess would be 8k based Sql Server's preferred page size.

Answer 3

A quick little benchmark based on the document referenced shows no increase in performance on my system greater than 128KB buffer size. Your mileage may vary, feel free to use the below.

        Stopwatch sw = new Stopwatch();
        Random rand = new Random();  // seed a random number generator
        int numberOfBytes = 2 << 22; //8,192KB File
        byte nextByte;
        for (int i = 1; i <= 28; i++) //Limited loop to 28 to prevent out of memory
        {
            sw.Start();
            using (FileStream fs = new FileStream(
                String.Format(@"C:\TEMP\TEST{0}.DAT", i),  // name of file
                FileMode.Create,    // create or overwrite existing file
                FileAccess.Write,   // write-only access
                FileShare.None,     // no sharing
                2 << i,             // block transfer of i=18 -> size = 256 KB
                FileOptions.None))  
            {
                for (int j = 0; j < numberOfBytes; j++)
                {
                    nextByte = (byte)(rand.Next() % 256); // generate a random byte
                    fs.WriteByte(nextByte);               // write it
                } 
            }
            sw.Stop();
            Console.WriteLine(String.Format("Buffer is 2 << {0} Elapsed: {1}", i, sw.Elapsed));
            sw.Reset();
        }