Problem with namespace while parsing XML file in C#

Question

I have this XML file of size 2.8GB (Polish Wikipedia dump). I have to search this file for certain title and get page content for it. I use LINQ to XML for simplicity:

var text = from el in StreamXmlDocument(filePath)
           where el.Element("title").Value.Contains(titleToSearch)
           select (string)el.Element("revision").Element("text");

and

private IEnumerable<XElement> StreamXmlDocument(string uri)
{
    //code made accoring to informations at MSDN website available at URL:
    //http://msdn.microsoft.com/en-us/library/system.xml.linq.xnode.readfrom.aspx
    using (XmlReader reader = XmlReader.Create(uri))
    {

        reader.MoveToContent();

        while (reader.Read())
        {
            switch (reader.NodeType)
            {
                case XmlNodeType.Element:
                    if (reader.Name == "page")
                    {
                        XElement el = XElement.ReadFrom(reader) as XElement;
                        el.DescendantsAndSelf().Attributes().Where(n => n.IsNamespaceDeclaration).Remove();
                        if (el != null)
                        {
                            yield return el;
                        }
                    }
                    break;
            }
        }
    }

So the problem is that this file contains a xmlns attribute in first element:

<mediawiki xmlns="http://www.mediawiki.org/xml/export-0.4/" (...) >

and when I run the code above I get error no reference to object at this line:

where el.Element("title").Value.Contains(titleToSearch)

When I manually delete that xmlns attribute everything works fine. I found somewhere in the Internet that this:

el.DescendantsAndSelf().Attributes().Where(n => n.IsNamespaceDeclaration).Remove();

should delete all xmlns attributes from elements. But it doesn't.

Answer 1

Well, welcome at SO then ;-)

In XML, a namespace declaration is saint. Removing it may well make the XML unusable, so I'd advice against it (and it's a huge task on a 2.8GB file!). Each name should be considered unique as in {namespace}elementname (i.e, both) whenever you deal with XML. Linq to XML accepts namespaces and you should use them:

XNamespace wiki = "http://www.mediawiki.org/xml/export-0.4/";

var text = from el in StreamXmlDocument(filePath)
           where el.Element(wiki + "title").Value.Contains(titleToSearch)
           select (string)el.Element(wiki + "revision").Element(wiki + "text");

(may be ignored, you do this already):
A note on the XML: Linq2XML will load the whole thing in memory, I believe, just like DOM, which will require about 4.5 times the size of the file. This may be problematic. Read this MSDN blog about streaming Linq to XML.

Answer 2

I believe you want:

XNamespace ns = "http://www.mediawiki.org/xml/export-0.4/";

var text = from el in StreamXmlDocument(filePath)
           where el.Element(ns+"title").Value.Contains(titleToSearch)
           select (string)el.Element(ns+"revision").Element(ns+"text");