views:

31

answers:

3

From the big log file (about like 2532910 lines), the lines that I am looking for are very few (like 10 or 12). What is the best way to match and read these lines? My code is in c#. Is there a way a reader/stream can read only a pattern matching data?

Thanks

+1  A: 

This may be a duplicate of http://stackoverflow.com/questions/994382/filtering-streams-in-c. Useful info there, anyway.

Steve Townsend
Thanks for pointing me to the link Steve.
"just derive your own from TextReader and override its ReadLine method so that the first call returns the line of text you need appended and subsequent calls just function as normal." Even if I overload,wouldnt it mean that I would still be reading the entire log, line by line?
There's no way to filter the data line by line without processing it all, unless you have prior knowledge of the locations of the lines you need. In that case, you would not need to do the matching logic. _Somebody_ has to read and discard the data you don't want, yes? I guess you could read it as binary data instead of line by line but at some point you have to go back to the line form. I'm sure that System.IO will buffer the text data anyway.
Steve Townsend
That helps. Thanks Steve!
A: 

No but you can read the stream line by line and use RegEx to find matching lines

vc 74
This is what I am doing now and I was looking for a faster solution. I am not up for reading blocks of data for the fear of losing some.
+1  A: 

to read such a big files the best way is to use streamReader.ReadLine()

just like this:

StreamReader sr = new StreamReader(@"path_to_log");

int lineNum = 1;
const int searchingLineNum = 10;
string line = string.Empty;

while (sr.Peek() != -1)
{
    line = sr.ReadLine();

    if (lineNum == searchingLineNum)
    {
        break;
    }
    lineNum++;
}

Console.WriteLine(line); // do what you want with this line (parse using Regex)
Sebastian Brózda
This is what I am doing now and I was looking for a faster solution.
if you want to really fast solution, move loggin to database :)
Sebastian Brózda