How to read data in certain format

Question

I have a log file that can get pretty big.

The information in my log file is in a certain format and I want to be retreiving them a seperate blocks of data.

For example,

This is the start.

Blah Blah

Blah Blah Blah Blah Blah Blah

Blah

This is the start.

Blah Blah

Blah Blah Blah Blah Blah Blah

Blah Blah Blah Blah Blah Blah

Blah Blah Blah Blah Blah Blah

Blah

I want to get information from the "this is the start" to before the start of next "this is the start". What is the best way to do this? My code is in c#.

Answer 1

The following code will split the file into chunks delineated by the "This is the start." line and call a callback method to process each chunk:

public static void ProcessInChunks(string inputFilename,
    string delimiter, Action<IEnumerable<string>> processChunk)
{
    using (var enumerator = File.ReadLines(inputFilename).GetEnumerator())
    {
        if (!enumerator.MoveNext())
            // The file is empty.
            return;

        var firstLine = enumerator.Current;
        if (firstLine != delimiter)
            throw new InvalidOperationException(
                "Expected the first line to be a delimiter.");

        List<string> currentChunk = new List<string>();

        while (enumerator.MoveNext())
        {
            if (enumerator.Current == delimiter)
            {
                processChunk(currentChunk);
                currentChunk = new List<string>();
            }
            else
                currentChunk.Add(enumerator.Current);
        }
        processChunk(currentChunk);
    }

Usage:

ProcessInChunks(@"myfile.log", "This is the start.",
    chunk => { /* do something here */ });

Answer 2

If you can't change the log creation process, the answer by @Timwi will work well. If you can adjust the log creation process, you could create new date-stamped log file names every time you want to write This is the start.. This will create multiple log files, but they will already be split in the desired way. Obviously if the text to find can change, this won't work.