How to do Streaming read of a large XML file in C# 3.5 ...

Question

How can you do a streaming read on a large XML file that contains a xs:sequence just bellow root element, without loading the whole file into a XDocument instance in memory.

Answer 1

I think it's not possible if you want to use object model (i.e. XElement\XDocument) to query XML. Obviously, you can't build XML objects tree without reading enough data. However you can use XmlReader class.

The XmlReader class reads XML data from a stream or file. It provides non-cached, forward-only, read-only access to XML data.

Answer 2

Heres is a howto: http://support.microsoft.com/kb/301228/en-us Just remember that you should not use XmlTextReader but instead XmlReader in conjunction with XmlReader.Create

Answer 3

I'm confused by the mention of the "xs:sequence" - this is a XML Schema element.

Are you trying to open a large XML Schema file? Are you open a large XML file that is based on that schema? Or are you trying to open a large XML file and validate it at the same time?

None of these situations should provide you with a problem using the standard XmlReader (or XmlValidatingReader).

Reading XML with XMLReader: http://msdn.microsoft.com/en-us/library/9d83k261(VS.80).aspx

Answer 4

Going with a SAX-style element parser and the XmlTextReader class created with XmlReader.Create would be a good idea, yes. Here's a slightly-modified code example from CodeGuru:

void ParseURL(string strUrl)
{
  try
  {
    using (var reader = XmlReader.Create(strUrl))
    {
      while (reader.Read())
      {
        switch (reader.NodeType)
        {
          case XmlNodeType.Element:
            var attributes = new Hashtable();
            var strURI = reader.NamespaceURI;
            var strName = reader.Name;
            if (reader.HasAttributes)
            {
              for (int i = 0; i < reader.AttributeCount; i++)
              {
                reader.MoveToAttribute(i);
                attributes.Add(reader.Name,reader.Value);
              }
            }
            StartElement(strURI,strName,strName,attributes);
            break;
            //
            //you can handle other cases here
            //
            //case XmlNodeType.EndElement:
            // Todo
            //case XmlNodeType.Text:
            // Todo
            default:
            break;
          }
        }
      }
      catch (XmlException e)
      {
        Console.WriteLine("error occured: " + e.Message);
      }
    }
  }
}

Answer 5

That code sample tries to turn XmlReader style code into SAX style code - if you're writing code from scratch I'd just use XmlReader as it was intended - Pull not Push.

Answer 6

I can't add a comment, since I just signed up but the code sample posted by Hirvox and currently selected as the answer has a bug in it. It should not have the new statement when using the static Create method.

Current: using (var reader = new XmlReader.Create(strUrl)) Fixed: using (var reader = XmlReader.Create(strUrl))