Most efficient way to jump through a file and read lines?

Question

I want to use a FileStream and seek from the beginning of the file while moving forward in the file .01% of the file size at a time.

So I want to seek to a position in the file, read the entire line, if it matches my criteria I am done. If not, I seek ahead another .01.

C# is OK but VB.NET preferred.

I used to do it something like this in VB6...

            FileOpen(1, CurrentFullPath, OpenMode.Input, OpenAccess.Read, OpenShare.Shared)
        Dim FileLength As Long = LOF(1)

        For x As Single = 0.99 To 0 Step -0.01
            Seek(1, CInt(FileLength * x))
            Dim S As String = LineInput(1)
            S = LineInput(1)
            filePosition = Seek(1)
            If filePosition < 50000 Then
                filePosition = 1
                Exit For
            End If
            V = Split(S, ",")
            Dim MessageTime As Date = CDate(V(3) & " " & Mid$(V(4), 1, 8))
            Dim Diff As Integer = DateDiff(DateInterval.Minute, MessageTime, CDate(RequestedStartTime))
            If Diff >= 2 Then
                Exit For
            End If
        Next

But I don't want to use FileOpen, I want to use a FileStream.

Any help is greatly appreciated!

Answer 1

what bout something like this (C# version):

using (var file = System.IO.File.OpenText(filename))
{
     while (!file.EndOfStream)
     {
          string line = file.ReadLine();
          //do your logic here
          //Logical test - if true, then break
     }
}

EDIT: VB version here (warning - from a C# dev!)

Using file as FileStream = File.OpenText(filename)
    while Not file.EndOfStream
         Dim line as string = file.ReadLine()
  ''//Test to break
  ''//exit while if condition met
    End While
End Using

Answer 2

I normally prefer vb.net, but C#'s iterator blocks are slowly winning me over:

public static IEnumerable<string> SkimFile(string FileName)
{
    long delta = new FileInfo(FileName).Length / 100;
    long position = 0;

    using (StreamReader sr = new StreamReader(FileName))
    {
        while (position < 100)
        {
            sr.BaseStream.Seek(position * delta, SeekOrigin.Begin);
            yield return sr.ReadLine();
            position++;
        }
    }
}

Put it in a class library project and use it from vb like this:

Dim isMatch as Boolean = False
For Each s As String in SkimFile("FileName.txt")
    If (RequestedDate - CDate(s.SubString(3,11))).Minutes > 2 Then
       isMatch = True
       Exit For
    End If
Next s

(I took some liberties with you criteria (assumed fixed-width values rather than delimited) to make the example easier)

Answer 3

There's an example on MSDN.

Edit in response to comment:

I must admit I'm a bit confused, as you seemed insistant on using a buffered FileStream, but want to read a file a line at a time? You can do that quite simply using a StreamReader. I don't know VB, but in C# it would be something like this:

using (StreamReader sr = File.OpenText(pathToFile)) 
{
    string line = String.Empty;
    while ((line = sr.ReadLine()) != null) 
    {
        // process line
    }
}

See http://msdn.microsoft.com/en-us/library/system.io.file.aspx.

Answer 4

This is a more or less direct conversion of your code, where we use FileStream.Position to specify where in the file to read:

Dim fileStream As New System.IO.FileStream(CurrentFullPath, IO.FileMode.Open, IO.FileAccess.Read)
Dim streamReader As New System.IO.StreamReader(fileStream)
For x As Single = 0.99 To 0 Step -0.01
  fileStream.Position = CLng(fileStream.Length * x)
  Dim S As String = streamReader.ReadLine()
  '... etc.
Next
'...
streamReader.Close()

Update: Here's a similar implementation with Using / End Using:

Using streamReader As System.IO.StreamReader = System.IO.File.OpenText(CurrentFullPath)
  For x As Single = 0.99 To 0 Step -0.01
    streamReader.BaseStream.Position = CLng(streamReader.BaseStream.Length * x)
    Dim S As String = streamReader.ReadLine()
    '... etc.
  Next
End Using