I have a process that reads an XML file. It uses the XMLTextReader class to do this as it is supposed to be a fast, forward only XML parser/reader.
Works just great with a 1 megabyte test file but comes to a complete halt when working on a 12 meg file in the live system.
Are there any solutions to this other than writing my own XML reader? That's not the end of the world but I would prefer to use available standard components if possible
I have had similar performance issues in the past. I traced it back to trying to remotely resolve against a DTD/schema. Are you doing this? Try setting XmlTextReader.XmlResolver to null if possible.
SAXExpat used to be really good. Expat is the XML parser, almost a reference implementation. I remember using it to read some synchronization XML files sent over a TCP connection, sometimes really big files (around 50mb) without any kind of problem. And that was 3/4 years ago, in .NET 1.1 and really crappy computers.
Depends what you do with what you get out of the reader. Are you putting it in an XML DOM, or any object model for that matter? That would make a big memory hit not matter what language or library you use.
Maybe it is flawed in 1.1, thought about trying out 2.0? I never used the XmlTextReader in my 1.1 days, so I can't vouch for it: but since 2.0 it is perfect.
I would be very surprised if the problem were in the XmlTextReader.
If you spend a few minutes to write a test program that creates an XmlTextReader and simply uses Read() to read through each node in the file until it gets to the end of the document, I bet you'll find that it zooms through your 12mb file like a hot knife through butter. That's the first thing I'd try if I were experiencing this problem.
Because once you've eliminated XmlTextReader as the source of the problem, you can focus your attention on what's actually causing it - which is, very probably, the code that processes the nodes that you're reading, not the code that reads the nodes.
Just one thought. Are you opening a database transaction for the length of the entire process? If so try it without the transaction or at least commit more often during the process.
I hate to recommend this, but if the software isn't sold or external, you could try bringing in the reader from Mono and see if that fixes your woes.