Using LINQ to query flat text files with fixed-length records?

Question

I've got a file filled with records like this:

NCNSCF1124557200811UPPY19871230

The codes are all fixed-length, and some of them link to other flat files (sort of like a relational database). What's the best way of querying this data using LINQ?

This is what I came up with intuitively, but I was wondering if there's a more elegant way:

var records = File.ReadAllLines("data.txt");
var table = from record in records 
            select new {    FirstCode = record.Substring(0, 2), 
                            OtherCode = record.Substring(18, 4) };

Answer 1

I don't think there's a better way out of the box.

One could define a Flat-File Linq Provider which could make the whole thing much simpler, but as far as I know, no one has yet.

Answer 2

For one thing I wouldn't read it all into memory to start with. It's very easy to write a LineReader class which iterates over a file a line at a time. I've got a version in MiscUtil which you can use.

Unless you only want to read the results once, however, you might want to call ToList() at the end to avoid reading the file multiple times. (This is still nicer than reading all the lines and keeping that in memory - you only want to do the splitting once.)

Once you've basically got in-memory collections of all the tables, you can use normal LINQ to Objects to join them together etc. You might want to go to a more sophisticated data model to get indexes though.