How could I read a very large text file using StreamReader?

Question

I want to read a huge .txt file and I'm getting a memory overflow because of its sheer size.

Any help?

    private void button1_Click(object sender, EventArgs e)
    {
        using (var Reader = new StreamReader(@"C:\Test.txt"))
        {
            textBox1.Text += Reader.ReadLine();
        }
    }

Text file is just:

Line1
Line2
Line3

Literally like that.

I want to load the text file to a multiline textbox just as it is, 100% copy.

Answer 1

Read and process it one line at a time, or break it into chunks and deal with the chunks individually. You can also show us the code you have, and tell us what you are trying to accomplish with it.

Here is an example: http://stackoverflow.com/questions/1028224/c-read-text-file-containing-data-delimited-by-tabs Notice the ReadLine() and WriteLine() statements.

EDIT: I believe TextBox is severely limited by the number of characters it can hold. You can try using the AppendText() method on a RichTextBox instead.

Answer 2

You'll get much faster performance with the following:

textBox1.Text = File.ReadAllText(@"C:\Test.txt");

It might also help with your memory problem, since you're wasting an enormous amount of memory by allocating successively larger strings with each line read.

Granted, the GC should be collecting the older strings before you see an OutOfMemoryException, but I'd give the above a shot anyway.

Answer 3

Firstly, the code you posted will only put the first line of the file into the TextBox. What you want is this:

using (var reader = new StreamReader(@"C:\Test.txt"))
{
    while (!reader.EndOfStream)
        textBox1.Text += reader.ReadLine();
}

Now as for the OutOfMemoryException: I haven't tested this, but have you tried the TextBox.AppendText method instead of using +=? The latter will certainly be allocating a ton of strings, most of which are going to be nearly the length of the entire file by the time you near the end of the file.

For all I know, AppendText does this as well; but its existence leads me to suspect it's put there to deal with this scenario. I could be wrong -- like I said, haven't tested personally.

Answer 4

First use a rich text box instead of a regular text box. They're much better equiped for the large amounts of data you're using. However you still need to read the data in.

// use a string builer, the += on that many strings increasing in size
// is causing massive memory hoggage and very well could be part of your problem
StringBuilder sb = new StringBuilder();

// open a stream reader
using (var reader = new StreamReader(@"C:\Test.txt"))
{
    // read through the stream loading up the string builder
    while (!reader.EndOfStream)
    {
       sb.Append( reader.ReadLine() );
    }
}

// set the text and null the string builder for GC
textBox1.Text = sb.ToString();
sb = null;