I am joining 3 xml logfiles with LINQ to XML by id, but when my logfiles become too big, LINQ to XML doesn't work anymore, so i have to use XmlReader.
But now I am asking myself how to do this?
Should i loop through every document for each id, or is there a more simple elegant solution?
Anybody?
The XMLReader has an affinity for one file. So you can only process one file at a time. I suppose in your scenario its possible to fire up 3 readers and load the files into memory. A possible problem I see here is how you would efficiently "join" the 3 files. Say the first id in the first record is located somewhere in the middle of the second and at the end of the third. In order to find them you would have to repeatedly traverse the xml. That doesnt make sense. Personally I would try to load the information into a db in relate the files there -a s tables. As db structures you have all the power of indexing available to quickly link data.